Dynamically creating video based on structured documents

ABSTRACT

An electronic system dynamically creates video based on a structured document by associating video clips with items in a structured document includes a server that is connected to a user terminal. The user terminal sends to the server, as a selected item, at least one item in a structured document selected by the user. The server receives the item sent by the user terminal, identifies the item in the structured document selected by the user, identifies at least one dependent item having a dependent relationship with the selected item, dynamically creates a video on the basis of at least one video clip associated with each identified item and at least one video clip associated with each identified dependent item, and sends the video for playback on the user terminal.

BACKGROUND

The present invention relates to a technique for dynamically creatingvideo based on structured documents.

Electric manuals are being increasingly provided in place of papermanuals. Therefore, users are learning how to operate devices andsoftware by viewing electronic manuals on screens.

However, when electronic manuals are presented in text form similar topaper manuals, it can be difficult for a user to learn how to operate adevice or to actually operate the device using an electronic manual.When there is a discrepancy between information displayed on the screenof a device (such as on the display device of a computer) and the textin the electronic manual corresponding to the information, the user maynot know where to look in the information displayed on the device andthe learning may not stick.

When an electronic manual includes video, the user has to find the videoat the spot that the user wants to view or learn more about, or fastforward to the spot that the user wants to view or learn more about.

Known prior art presents a device in which notes can be selected fordisplay as guidance in accordance with user instructions entered byselecting functions. The device searches a guidance storage means forall of the notes corresponding to the selected functions, selects thenotes to be displayed as guidance in accordance with user instructionsfrom the notes in the search results, searches a guidance database foranimation data corresponding to the selected notes, and displaysanimation based on the animation data in the search results.

Known prior art describes an interactive interface method which is ableto realize an interactive interface between an operator and a machine.In this interactive interface method, the operation instructionprocedure is changed from the machine to the operator based on theoperator's degree of proficiency.

Known prior art describes a mobile communication device including: astorage unit for storing data entered by the user; a database unitstoring the appropriate operating instructions; a guidance unit formatching data in the database unit to data entered into the storageunit, estimating the user's intention for performing an operation, anddisplaying the estimated operating procedure on a display means; aguidance information supplying unit for supplying operational guidanceinformation to the guidance unit based on the operating procedureestimated by the guidance unit; and a determining unit for activatingthe guidance unit when data inputted by the user matches a predeterminedreference value and it has been determined that activation of theguidance unit is required.

Known prior art describes a maintenance guidance information that isstored in an information processing device, and the maintenance guidanceinformation is dynamically displayed on the screen of the display devicewhen the device cover is opened. More appropriate maintenance guidanceinformation can then be displayed in response to an instruction enteredusing an input device.

Known prior art describes a device that includes: an operation storageunit for storing a series of operations performed to recover from amalfunction; a display unit for displaying these operations; and acontrol unit for executing an operation displayed on the display unit onthe basis of information entered using a mouse where the mouse is usedto operate a malfunction recovery guidance function, and the operationsstored in the storage unit are displayed on the display unit in apredetermined order.

SUMMARY

In an embodiment of the present invention, a method and/or computerprogram product dynamically creates video based on a structured documentby associating video clips with items in a structured document. At leastone item in the structured document selected by a user is identified,and at least one dependent item having a dependent relationship witheach identified item is identified. A video is dynamically created basedon at least one video clip associated with each identified item and atleast one video clip associated with each identified dependent item. Thevideo is then provided to a user.

In an embodiment of the present invention, an electronic system fordynamically creating video based on a structured document by associatingvideo clips with items in a structured document includes a server thatis connected to a user terminal. The user terminal sends to the server,as a selected item, at least one item in a structured document selectedby the user. The server receives the item sent by the user terminal,identifies the item in the structured document selected by the user,identifies at least one dependent item having a dependent relationshipwith the selected item, dynamically creates a video on the basis of atleast one video clip associated with each identified item and at leastone video clip associated with each identified dependent item, and sendsthe video for playback on the user terminal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram showing an example of an electronic device (a userterminal or a server) that can be used according to an embodiment of thepresent invention.

FIG. 1B is a diagram showing an example of a user terminal that can beused according to an embodiment of the present invention.

FIG. 2A shows an example of a user interface enabling a user to selectone or more particular items in an electronic manual.

FIG. 2B shows an example of a user interface enabling a user to selectone or more particular items in an electronic manual.

FIG. 2C shows an example of a user interface enabling a user to selectone or more particular items in an electronic manual.

FIG. 2D shows an example of a user interface enabling a user to selectone or more particular items in an electronic manual.

FIG. 3A shows an example of a dependent item having a dependentrelationship with a user-specified item being identified according to anembodiment of the present invention.

FIG. 3B shows an example of video clips associated with bothuser-specified items and dependent items being identified according toan embodiment of the present invention.

FIG. 3C shows an example of video clips in a video classified accordingto different categories (“main step”, “transitional step” and“supplemental step”) according to an embodiment of the presentinvention.

FIG. 4A shows an example of a user interface for classifying video clipsconstituting the video according to the “transitional step” categoryaccording to an embodiment of the present invention.

FIG. 4B shows an example of a user interface for classifying video clipsconstituting the video according to the “transitional step” categoryaccording to an embodiment of the present invention.

FIG. 5A shows an example of video clips associated with user-selecteditems and dependent items, and an example of a video composed of thesevideo clips according to an embodiment of the present invention.

FIG. 5B shows an example in which some of the video clips constitutingthe video and classified according to the “supplemental step” categoryare removed according to an embodiment of the present invention.

FIG. 5C shows an example in which the video clips constituting the videoand classified according to the “supplemental step” category are playedback faster than the video clips classified according to the “main step”and “transitional step” categories.

FIG. 6A shows an example of a dependent item having a dependentrelationship with a user-specified item being identified according to anembodiment of the present invention different from the embodiment shownin FIG. 3A.

FIG. 6B shows an example of video clips associated with bothuser-specified items and dependent items being identified according toan embodiment of the present invention different from the embodimentshown in FIG. 3B.

FIG. 6C shows an example of video clips in a video classified accordingto different categories (“main step”, “transitional step” and“supplemental step”) according to an embodiment of the present inventiondifferent from the embodiment shown in FIG. 3C.

FIG. 7A shows an example of a dependent item having a dependentrelationship with a user-specified item being identified according to anembodiment of the present invention different from the embodiment shownin FIG. 3A.

FIG. 7B shows an example of video clips associated with bothuser-specified items and dependent items being identified according toan embodiment of the present invention different from the embodimentshown in FIG. 3B.

FIG. 7C shows an example of video clips in a video classified accordingto different categories (“main step”, “transitional step” and“supplemental step”) according to an embodiment of the present inventiondifferent from the embodiment shown in FIG. 3C.

FIG. 8 shows the timing for the process of creating a video from videoclips (the video clip consolidation process) and the process of playingback the video clips in the video.

FIG. 9A is a general flowchart of the process of dynamically creatingvideos on the basis of a structured document according to an embodimentof the present invention.

FIG. 9B shows an example of a flowchart for creating a step ID list inthe process of dynamically creating a video on the basis of a structureddocument according to an embodiment of the present invention.

FIG. 9C shows an example of a flowchart for the process of resolvingdependent items and creating a step ID list in the flowchart forcreating a step ID list shown in FIG. 9B.

FIG. 9D shows an example of a flowchart for the process of determiningthe insertion locations for transitional steps in the step ID listcreated in accordance with the processing shown in the flowchart of FIG.9C and inserting the transitional steps in the flowchart for creating astep ID list shown in FIG. 9B.

FIG. 9E shows an example of a flowchart for selecting video clips andcreating a track in the process of dynamically creating a video on thebasis of a structured document according to an embodiment of the presentinvention.

FIG. 9F shows an example of a flowchart for creating a video from atrack in the process of dynamically creating a video on the basis of astructured document according to an embodiment of the present invention.

FIG. 10A shows a step ID list that has been consolidated according tothe processing shown in the flowchart of FIG. 9C.

FIG. 10B shows the insertion of transitional steps into a step ID list(1029) with all dependencies resolved according to the processing shownin the flowchart of FIG. 9D.

FIG. 11A is a functional block diagram of an electronic devicepreferably including a hardware configuration according to FIG. 1 fordynamically creating video on the basis of a structured document inaccordance with an embodiment of the present invention in which theelectronic device is a user terminal.

FIG. 11B is a functional block diagram of an electronic devicepreferably including a hardware configuration according to FIG. 1 fordynamically creating video on the basis of a structured document inaccordance with an embodiment of the present invention in which theelectronic device is a server connected to a user terminal.

DETAILED DESCRIPTION

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The following is an explanation of an embodiment of the presentinvention with reference to the drawings. In the drawings describedbelow, the same reference signs are used to denote the same elementsunless otherwise noted. It should be understood that the embodiment ofthe present invention is used to explain a preferred embodiment of thepresent invention and that there is no intention to limit the scope ofthe present invention to the embodiment shown herein.

The following is an explanation of the terminology used in an embodimentof the present invention.

In an embodiment of the present invention, a “structured document” is anelectronic document that can be rendered as a list of items. Forexample, a structured document may be realized as a tree structure. Whena structured document is realized as a tree structure, each item is anode in the tree structure. Structured documents include XML and HTMLdocuments. Other examples of structured documents include menu documentsfor playing back video (for example, playback menu documents on DVDdisks, Blu-ray disks or hard disk recorders), electronic manuals,electronic instruction manuals, electronic operating manuals, electronichelp documents, and electronic tutorial documents. Structured documentssometimes accompany products and services (on CD-ROMs or memorydevices), are incorporated into products (on a memory device inside theproduct), or are downloaded via the internet or an intranet. The productmay be a product for commercial sale or for use in-house. Examplesinclude, but are not limited to, computer hardware and software, devices(IT devices and testing devices), home electronics, and educationalmaterials. Services include, but are not limited to, real estateproducts (house and condominium sales), financial products, securitiesproducts, insurance products (life insurance, casualty insurance,earthquake insurance and fire insurance), travel products (such asplanned travel), loan products (homeowner loans and car loans),ticketing products (movies, sports and the arts), educational products(language classes, training classes and seminars), and data products(such as personal data).

In an embodiment of the present invention “item” may be used as a tableof contents feature. “Item” may refer to the table of contents, aheader, a title or a step.

In an embodiment of the present invention, a “dependent item having adependent relationship with an item” may be an item including anexplanation or background information associated with an item selectedby the user, or may be an item omitted from the explanation associatedwith an item selected by the user.

In an embodiment of the present invention, a “video” is composed of aplurality of video clips. Videos are dynamically created by theelectronic device.

In an embodiment of the present invention, a “video clip” is the videodata associated with each item. A video clip is video data preparedbeforehand, such as a prerecorded video clip. In an embodiment of thepresent invention, a video clip may be video data obtained by an“operation script” run on a user terminal. An operation script is agroup of instructions given to the user terminal. An operating scriptrun on the user terminal can dynamically execute an operation to beperformed by the user (for example, a mouse operation, keyboardoperation or input operation) in a simulated way to show the user how toperform the operation. The user terminal may execute an operation scripton a virtual execution screen separate from the screen currently beingused by the user. The virtual execution screen may be an emulatedscreen, a simulated screen or a pseudo-execution screen. By running anoperation script on the user terminal, a user can perform an on-the-spotverification of an actual operation to be performed on the userterminal. When the phrase “video clip” is used below, it should beunderstood to include video data obtained by running an operation scriptunless otherwise indicated.

In an embodiment of the present invention, “tracks” refer to stepnumbers selected by the user or inserted as a result of resolvingdependent relationships with selected step numbers. Video clip IDs,script IDs and user-selected information associated with these stepnumbers (such as the clip categories “main step”, “transitional step” or“supplemental step”) are arranged in playback order using the stepnumbers. This data is analogous to the screenplay of a movie.

In the present embodiment, “electronic device” may refer to a serverable to receive items selected by a user using a user terminal or mayrefer to a user terminal including a structured document.

In an embodiment of the present invention, the user terminal is notrestricted to a particular web browser application or application usedto display the help function. Examples of web browsers that can be usedinclude Internet Explorer (registered trademark) (IE), Google Chrome(registered trademark), Safari (registered trademark), and Firefox(registered trademark). The electronic device may be a personal computersuch as a desktop computer or notebook computer, an internet television,a gaming console, a tablet, a smartphone (such as the tablet orsmartphone in which Windows (registered trademark), Android (trademark)or iOS (trademark) may be installed), or any other type of device.

FIG. 1A and FIG. 1B are diagrams showing an example of hardware(electronic device, user terminal) used in an embodiment of the presentinvention.

FIG. 1A is a diagram showing an example of a device (server or userterminal) used in an embodiment of the present invention.

This electronic device (101) includes a central processing unit (CPU)(102) and a main memory (103), and these are connected to a bus (104).The CPU (102) is preferably based on a 32-bit or 64-bit architecture.Examples that can be used include the Core i (trademark) series, Core 2(trademark) series, Atom (trademark) series, Xeon (registered trademark)series, Pentium (registered trademark) series or Celeron (registeredtrademark) series from Intel Corporation, the A (trademark) series,Phenom (trademark) series, Athlon (trademark) series, Turion (registeredtrademark) series or the Sempron (trademark) series from Advanced MicroDevices, and the Power (trademark) series from International BusinessMachines Incorporated.

A display (106) such as a liquid crystal display (LCD) may be connectedto the bus (104) via a display controller (105). The display (106) maybe used to display objects via a suitable graphic interface by runningsoftware on the electronic device (101) (for example, the electronicdevice program in an embodiment of the present invention or any otherelectronic device program run on the electronic device (101) such as anyother type of computer program). The display (106) may output screensfrom a web browser application.

A disk (108) such as a hard disk or solid state drive (SSD) may beoptionally connected to the bus (104) via an S-ATA or IDE controller(107).

A drive (109) such as a CD, DVD or BD drive may be optionally connectedto the bus (104) via a SATA or IDE controller (107).

A keyboard (111) and mouse (112) may be optionally connected to the bus(104) via a peripheral device controller (110) such as a keyboard/mousecontroller or USB bus.

An operating system such as Windows (registered trademark), UNIX(registered trademark), MacOS (registered trademark), a Java (registeredtrademark) processing environment such as J2EE, Java (registeredtrademark) applications, Java (registered trademark) virtual machines(VM), and Java (registered trademark) just-in-time (JIT) compilers, theelectronic device program according to the embodiment of the presentinvention, as well as other electronic device programs and data, may bestored for loading into the main memory (103).

The display (108) may be built into the electronic device (101),connected via a cable so as to be accessible by the electronic device(101), or connected via a wired or wireless network so as to beaccessible by the electronic device (101).

If necessary, a drive (109) may be used to install a program such as anoperating system, application or an electronic device program accordingto an embodiment of the present embodiment on the disk (108) from aCD-ROM, DVD-ROM or BD.

The communication interface (114) may run according to the Ethernet(registered trademark) protocol. The communication interface (114) isconnected to a bus (104) via a communication controller (113), andfunctions to physically connect the electronic device (101) to thecommunication network (115) via wires or wirelessly. This provides anetwork interface layer for the TCP/IP communication protocol in thecommunication function of the operating system of the electronic device(101). The communication line can operate in a wireless LAN environmentor a Wi-Fi wireless LAN environment according to wireless LAN connectionstandards such as IEEE 802.11a/b/g/n, or a mobile phone networkenvironment (such as a 3G or 4G environment).

FIG. 1B is a diagram showing an example of a user terminal that may beused in an embodiment of the present invention.

In the electronic device (121) shown in FIG. 1B, the CPU (122), mainmemory (123), bus (124), display controller (125), display (126), SSD(128), communication controller (133), communication interface (134) andcommunication network (135) correspond to the CPU (102), main memory(103), bus (104), display controller (105), display (106), SSD (108),communication controller (113), communication interface (114) andcommunication network (115) in the electronic device (101) shown in FIG.1A.

When the electronic device (121) is a smartphone, mobile phone ortablet, the CPU (122) may be any CPU commonly used in a smartphone,mobile phone or tablet, or the A Series (registered trademark) fromApple, Inc.

The smartphone OS (such as the Android OS (registered trademark), theWindows Phone OS (registered trademark), Windows (registered trademark)or iOS), the application program according to an embodiment of thepresent invention, other programs, and data may be stored on the SSD(128) disk for loading into the main memory (123).

The keyboard realizing means (130) may be an application for displayinga software keyboard on the display (126).

FIG. 2A through FIG. 2D, FIG. 3A through FIG. 3C, FIG. 4A through FIG.4D, FIG. 5A through FIG. 5B, FIG. 6A through FIG. 6C, FIG. 7A throughFIG. 7C and FIG. 8 show the series of operations performed in accordancewith an embodiment of the present invention to dynamically create videoon the basis of a structured document: identifying at least one item inthe structured document selected by the user (FIG. 2A through FIG. 2Band FIG. 2C through FIG. 2D); identifying at least one item having adependent relationship with the identified item (FIG. 3A, FIG. 6A andFIG. 7A); selecting video clips associated with the identified items andcreating a video (FIG. 3B, FIG. 6B, FIG. 6C, FIG. 7B and FIG. 7C);categorizing each video clip in the created video (FIG. 3C, FIG. 6C andFIG. 7C); and performing processing to play back the created video (FIG.4A through FIG. 4B, FIG. 5A through FIG. 5B, FIG. 6C and FIG. 7C). FIG.5C shows the operations performed to play back the video created inaccordance with an embodiment of the present invention (FIG. 5C).

FIG. 2A through FIG. 2B and FIG. 2C through FIG. 2D show an example of auser interface in an embodiment of the present invention which allowsthe user to select one or more specific items in an electronic manual.

FIG. 2A through FIG. 2B show an example of a user interface in anembodiment of the present invention which allows the user to select oneor more specific items in an online electronic manual for a softwareapplication.

The screen (201) shown in FIG. 2A is an online manual screen for asoftware application that is being displayed on the display device of auser terminal. The user views the online manual on the screen (201). Theuser interface for the online manual includes a table of contents panedisplaying the table of contents, and an explanation displaying pane(203) for displaying the explanation of each item in the table ofcontents. The online manual screen (201) includes a “switch to selectionscreen” button (204) which allows the user to switch to the selectionscreen in order to select one or more items in the online manual.Alternatively, the online manual screen (201) may display a keyboardshortcut that can be activated by pressing a specific key on thekeyboard to switch to the selection screen and allow the user to selectone or more items in the online manual (for example, “Press F9 to switchto the selection screen”). The user may also switch to the selectionscreen to select one or more items in the online manual by performing acertain gesture. Possible gestures include: (1) sliding one or morefingers from the left or right side of the screen or a track pad towardsthe center; (2) tracing a circle on the screen or track pad using one ormore fingers; or (3) moving one or more fingers on both the left andright hands in opposite directions vertically or horizontally on thescreen or track pad (for example, moving two fingers on the right handupwards and two fingers on the left hand downwards).

In this example, the user wishes to switch to the selection screen inorder to select one or more items in the online manual. The user clicksthe “switch to the selection screen” button (204). The user may alsoselect and click on “switch to selection mode” instruction in the menu.The user may alternatively perform a certain action to switch to theselection screen, such as pressing a specific key on the keyboard orperform a certain gesture on the user terminal. The user terminaldisplays the selection screen (211) shown in FIG. 2B on the displaydevice of the user terminal in response to the user pressing the “switchto the selection screen” button (204) (or clicking on “switch toselection mode” or performing a certain action.

The selection screen (211) shown in FIG. 2B includes a table of contentspane (212) for the table of contents screen (202), and an explanationdisplay pane (213) for the explanation display screen (203). Each itemin the table of contents pane (212) has a button (or check box denotedby □) enabling the item to be selected. Each item in the explanationdisplay pane (213) also has a button (or check box denoted by □)enabling the item to be selected. The selection screen (211) alsoincludes a button for returning to the online manual screen (201) (thatis, a “switch to normal screen” button (214) for switching to the“normal mode”). Alternatively, the selection screen (211) may display akeyboard shortcut that can be activated by pressing a specific key onthe keyboard to switch to the online manual screen (201) (for example,“Press F9 to switch to the normal screen”), or the user may perform acertain gesture to switch to the online manual screen (201). Thisgesture may be any one of gestures (1) through (3) described above.

Alternatively, there may be neither a “normal mode” nor a “selectionmode”. Instead, the button for selecting items may be displayed on thescreen to the left of the items so that the user can click on the textof the item to move to the page associated with the text of the clickeditem (corresponding to the “normal mode” described above), or can clickon the button to create a video associated with the item (correspondingto the “selection mode” described above).

When the user has selected all of the items (in the table of contents)in the table of contents pane (212) that the user wishes to view or hasan interest in (referred to as a “batch selection” below), the user mayclick on the first item (referred to as the “item in the start position”below) (221) (for example, turning on the check box) while holding downa specific key (such as the Ctrl key). Next, the user may click on thelast item (referred to as the “item in the end position” below) (222)(for example, turning on the check box) that the user wishes to view orhas an interest in while holding down a specific key (such as the Ctrlkey). In response to the item in the start position (221) and the itemin the end position (222) being clicked while a specific key was beingheld down (such as the Ctrl key), the user terminal may dynamicallyselect all of the items between the item in the start position (221) andthe item in the end position (222), and displays the check boxes of thedynamically selected items in an intermediate state between on and off.Next, the user may again use the table of contents pane (212) to selectan item in the start position and an item in the end position differentfrom the item in the start position (221) and the item in the endposition (222) selected earlier. The user may also click on a singleitem (or turn on a check box) without holding down a specific key toindividually select an item that the user wishes to view or has aninterest in (referred to as “individual selection” below).

Also, when the user has selected all of the items in the explanationdisplay pane (213) that the user wishes to view or has an interest in(that is, has performed a “batch selection”), the user may click on thefirst item (the “item in the start position”) (231) (for example,turning on the check box) while holding down a specific key (such as theCtrl key). Next, the user may click on the last item (the “item in theend position”) (232) (for example, turning on the check box) that theuser wishes to view or has an interest in while holding down a specifickey (such as the Ctrl key). In response to the item in the startposition (231) and the item in the end position (232) being clickedwhile a specific key was being held down (such as the Ctrl key), theuser terminal may dynamically select all of the items between the itemin the start position (231) and the item in the end position (232), anddisplays the check boxes of the dynamically selected items in anintermediate state between on and off. Next, the user may again use theexplanation display pane (213) to select an item in the start positionand an item in the end position different from the item in the startposition (231) and the item in the end position (232) selected earlier.The user may also click on a single item (or turn on a check box) in theexplanation display pane (213) without holding down a specific key toindividually select an item (for example, the table of contents) thatthe user wishes to view or has an interest in (that is, perform an“individual selection”).

In response to the user selecting items combined with the item in thestart position and the item in the end position, the user terminal mayidentify the items in the online manual selected by the user.

Instead of selecting a combination of items including an item in thestart position and an item in the end position, the user may select asingle item (such as a chapter title in the table contents or a titlereferring to all of the tasks) (for example, by turning on the check boxfor a large item) (not shown). Examples of chapter titles in the tableof contents pane (212) shown in FIG. 2B include “Initial Job”,“Sketching a Job Design”, “Setup for Data Connection”, “Defining Data”and “Making Jobs Adjustable”. An example of a level showing all of theindividual tasks is “Creating a New Job” shown in the explanationdisplay pane (213) shown in FIG. 2B.

The user terminal may respond to the user selecting a large item such asan entire chapter or an entire section by identifying the item selectedby the user in the online manual.

FIG. 2C through FIG. 2D show an example of a user interface in anembodiment of the present invention which allows the user to select oneor more items in an electronic manual.

The screens (241, 251) shown in FIG. 2C and FIG. 2D are screens showingall of the steps in an electronic manual for software. There are 100steps ranging from Step 1-1 to Step 10-10. Each step is assigned anumber. The title of each step has been omitted from the screens (241,251).

First, the user clicks the check box of the first step in the electronicmanual in which the user is interested (Step 3-3) and then clicks thecheck box of the final step in which the user is interested (Step 6-6).In response to the user clicking on Step 3-3 and Step 6-6, theelectronic device dynamically selects all of the items between Step 3-3and Step 6-6 (Step 3-4 through Step 6-5), and displays the check boxesof the dynamically selected items in an intermediate state between onand off (see screen (241)). The electronic device then identifies Step3-3 through Step 6-6 as items selected by the user.

Next, the user clicks on the last step in the electronic item in whichthe user is interested (Step 9-3) and then clicks on the first step inwhich the user is interested (Step 8-2). In response to the userclicking on Step 9-3 and Step 8-2, the electronic device dynamicallyselects all of the items between Step 8-2 and Step 9-3 (Step 8-3 throughStep 9-2) because this is the sequence of numbers between the stepnumber of the first step clicked on and the step number of the last stepclicked on. The check boxes of the dynamically selected items aredisplayed in an intermediate state between on and off (see screen(251)). The electronic device identifies Step 8-2 through Step 9-3 asitems selected by the user.

The screens (241, 251) in FIG. 2C and FIG. 2D show all of the steps inthe electronic manual. Alternatively, the user terminal may display someof the steps on the screen (in the window), and display the remainingsteps as the user scrolls down the screen using a scroll bar.

FIG. 3A is an example showing how dependent items having a dependentrelationship with user-selected items are identified in an embodiment ofthe present invention.

The step ID list (301) shown in FIG. 3A links each item selected by theuser in FIG. 2C through FIG. 2D, that is, Steps 3-3 through Step 6-6(311) and Step 8-2 through Step 9-3 (312).

The electronic device may identify any dependent item having a dependentrelationship with the items in the step ID list (301) using thedependent item identifying table (321) which lists items and dependentitems having a dependent relationship with those items. The dependentitem identifying table (321) may include task IDs (which are IDsassociated with items and are IDs ranked higher than step IDs), step IDsassociated with task IDs (which are IDs associated with items), anddependency IDs (which are IDs associated with dependent items having adependent relationship with items having step IDs). The dependent itemidentifying table (321) shows that the steps with step IDs 3-3 through3-10 have a dependency relationship with step IDs 2-x (that is, Step 2-1through Step 2-10). Similarly, the dependent item identifying table(321) shows that the steps with step IDs 9-1 through 9-10 have adependency relationship with step IDs 7-x (that is, Step 7-1 throughStep 7-10). The dependent item identifying table (321) may assume theformat of a table or any other data format as long as the IDs areassociated with dependency IDs.

The electronic device uses the dependent item identifying table (321) todetermine that, among Step 3-3 through Step 6-6 (311), Step 3-3 throughStep 3-10 have dependency relationships with Steps 2-x (that is, Step2-1 through Step 2-10) (331).

Similarly, the electronic device uses the dependent item identifyingtable (321) to determine that, among Step 8-2 through Step 9-3 (312),Step 9-1 through Step 9-3 have dependency relationships with Steps 7-x(that is, Step 7-1 through Step 7-10) (332).

The electronic device may insert each dependent item having a dependencyrelationship with Step 3-3 through Step 3-10 and Step 9-1 through Step9-3, that is, Step 2-1 through Step 2-10 (351) and Step 7-1 through Step7-10 (352) in a step ID list (301) according to the flowchart for theprocess of creating a step ID list shown in FIG. 9B through FIG. 9D to acreate step ID list (341). The step ID list (341) has Step 2-1 throughStep 2-10 (351), Step 3-3 through Step 6-6 (311), Step 8-2 through Step8-10 (312 a), Step 7-1 through Step 7-10 (352), and Step 9-1 throughStep 9-3 (312 b). In this example, Step 2-1 through Step 2-10 (351) areinserted in front of Step 3-3 through Step 6-6 (311), and Step 7-1through Step 7-10 (352) are inserted between Step 8-2 through Step 8-10(312 a) and Step 9-1 through Step 9-3 (312 b) among Step 8-2 throughStep 9-3 (312).

Alternatively, the electronic device may insert each of the identifieddependent items having a dependent relationship with Step 3-3 throughStep 3-10 and Step 9-1 through Step 9-3, that is, Step 2-1 through Step2-10 and Step 7-1 through Step 7-10, in the step ID list (301), and thencreate a step ID list (not shown) by sorting according to step number orby referencing the dependency relationship flags showing the dependencyrelationships between the steps.

FIG. 3B shows an example in which video clips associated withuser-selected items and dependent items in the step ID list (341) areidentified in an embodiment of the present invention.

The electronic device may identify video clips associated with each itemin the step ID list (341) using video clip identifying table (361) orvideo clip identifying table (362) including step IDs (the IDscorresponding to item IDs), video clip IDs associated with the step IDs,or script IDs.

Both video clip identifying table (361) and video clip identifying table(362) include step IDs (IDs corresponding to items), and video clip IDsand script IDs associated with step IDs. Video clip identifying table(361) and video clip identifying table (362) may assume the format of atable or any other data format as long as video clip IDs or script IDsare associated with step IDs.

In video clip identifying table (361) and video clip identifying table(362), either a video clip ID or script ID is set for each step ID.Alternatively, in video clip identifying table (361) and video clipidentifying table (362), both a video clip ID and a script ID may be setfor each step ID. When both a video clip ID and a script ID are set foreach step ID in video clip identifying table (361) and video clipidentifying table (362), the user either selects video clip IDs orscript IDs during video selection, or a priority is established betweenvideo clip IDs and script IDs in video clip identifying table (361) andvideo clip identifying table (362).

In video clip identifying table (361), the video clip IDs and script IDsmay be denoted using alphanumeric identifiers. There are no particularrestrictions on the type of alphanumeric characters that are used(uppercase letters, lowercase letters, or numerals) or the number ofcharacters used as long as the uniqueness of each sequence can beensured. For example, if identifiers composed of four columns ofuppercase letters and numerals are used, approximately 1.68 (36⁴)million unique identifiers can be used

In video clip identifying table (362), the video clip IDs and script IDsmay be denoted using URLs as identifiers which allow video clips orscripts to be downloaded or accessed. These URL identifiers can take theform of an ID such as “ABCX” at the end of a URL, such as“http://example.com/scripts/ABCX”. Alternately, these URL identifierscan be defined using an independent protocol scheme, such as“dmcp://example.com/scripts/ABCX” (where dmcp=Document and Movie ControlProtocol). A file on a computer can be referenced using“file:///Users/admin/Documents/scripts/ABCX.bat”, and a uniform resourceidentifier (URI) superordinate to a URL can be used as an ID for clipsand scripts by various providers instead of not indicating an addressdirectly, such as “cid:ABCX@example.com”.

The electronic device uses the video clip identifying table (361) toacquire the video clip ID or script ID associated with the step IDcorresponding to a step number in the step ID list (341), and associatesthe acquired video clip ID or script ID with the step in the step IDlist (341). For example, the electronic device associates script ID“3R4P” with Step 2-1 through Step 2-4 (351) in the step ID list (341),associates script ID “FR4T” with the Step 2-5 through Step 2-6 (351),and associates script ID “E34R” with Step 2-7 through Step 2-10 (351).The electronic device associates video clip IDs or script IDs with Step3-3 through Step 6-6 (311), Step 8-2 through Step 8-10 (312 a), Step 7-1through Step 7-10 (352), and Step 9-1 through Step 9-3 (312 b) in thesame way. The electronic device can dynamically create a provisionalvideo by linking the video clips associated with the associated videoclip IDs or script IDs.

Similarly, the electronic device uses the video clip identifying table(362) to acquire the video clip ID or script ID associated with the stepID corresponding to a step number in the step ID list (341), andassociates the acquired video clip ID or script ID with the step in thestep ID list (341). The electronic device can dynamically create aprovisional video by linking the video clips associated with theassociated video clip IDs or script IDs.

FIG. 3C shows an example in which the video clips in a video areclassified by category (“main step”, “transitional step” and“supplemental step”) in an embodiment of the present invention.

The step ID list (341) is the same step ID list (341) shown in FIG. 3A,and has Step 2-1 through Step 2-10 (351), Step 3-3 through Step 6-6(311), Step 8-2 through Step 8-10 (312 a), Step 7-1 through Step 7-10(352) and Step 9-1 through Step 9-3 (312 b). Video clips are associatedwith each step using the process shown in FIG. 3B.

The electronic device sets the category of the video clips associatedwith items in the step ID list (341) identified by the user (Step 3-3through Step 6-6 (311), Step 8-2 through Step 8-10 (312 a), and Step 9-1through Step 9-3 (312 b)) to “main step”.

The electronic device sets the category of the video clips associatedwith other items in the step ID list (341) identified by the user (Step2-1 through Step 2-10 (351) and Step 7-1 through Step 7-10 (352)) to“supplemental step”.

The electronic device sets the category of the video clips associatedwith items directly adjacent to items and dependent items in the step IDlist (341) identified by the user (the set Step 2-10 through Step 3-3(353), the set Step 8-10 through Step 7-1 (354) and the Step 7-10through Step 9-1 (355)) to “transitional step”. When the beginning stepin the step ID list (341) is a supplemental step, the category of thevideo clip associated with Step 2-1 (356) is also set to “transitionalstep” because a transitional step has to be played at the video startpoint. A transitional step is a portion corresponding to a scene change.When a video clip is set as a transitional step, the transitional stepmay be added to a supplemental step/main step/main step/supplementalstep combination or supplemental step/supplemental step combination.When the beginning of the step ID list (341) is a supplemental step, itmay be a transitional step (the start step played at the video startpoint)/supplemental step combination.

In the categorized track (371), the two categories of supplemental stepand transitional step have been applied, respectively, to the videoclips associated with the beginning of a supplemental step (Step 2-1,Step 7-1), and to the video clips associated with the end of asupplemental step (Step 2-10, Step 7-10). Similarly, when a main stepand a supplemental step are adjacent to each other, the two categoriesof main step and transitional step have been applied, respectively, tothe video clips associated with the beginning of a main step (Step 3-3,Step 9-1), and to the video clip associated with the end of a main step(Step 8-10).

In the categorized track (371), the video clips associated with eachitem in the categorized track (371) were categorized as follows:transitional step 1 (385), supplemental step 1 (383), transitional step2 (386), main step 1 (381), transitional step 3 (387), supplemental step2 (384), transitional step 4 (388), and main step 2 (382).

In the categorized track (371), video clips are associated with each inthe categorized steps: transitional step 1 (385), supplemental step 1(383), transitional step 2 (386), main step 1 (381), transitional step 3(387), supplemental step 2 (384), transitional step 4 (388), and mainstep 2 (382). As a result, the electronic device may dynamically createa video by linking transitional step 1 (385), supplemental step 1 (383),transitional step 2 (386), main step 1 (381), transitional step 3 (387),supplemental step 2 (384), transitional step 4 (388), and main step 2(382).

By setting each video clip according to category (“main step”,“transitional step”, “supplemental step”), a video can be played back indifferent ways by category. The following are examples of playback bycategory.

The electronic device plays the video back at 1× speed in a main step.

In a transitional step, the electronic device may acquire the item(chapter, section, etc.) to be explained in the main step orsupplemental step following the transitional step, and display the titleor summary associated with the item. In a transitional step, theelectronic device may also display a short explanatory passage such as“Item Y, on which Item X is premised, will now be explained” for a setperiod of time.

In a supplemental step, the electronic device affixes flags for playbackaccording to attributes described below. Playback information for videoclips categorized as supplemental steps includes (time, playback speed,capture information, and display method such as display range anddisplay position. Attributes include the default or user-establishedplayback time (for example, 30 seconds or 60 seconds), playback speed(slower than 1×, or 2×, 4× or 6×), playback-or-skip, and playback windowsize.

For example, the electronic device may play back a video clip associatedwith a supplemental step faster than 1× (or the full playback speed) inaccordance with playback information affixed to the video clip.

In a supplemental step, the electronic device may also play back thevideo clip categorized as a supplemental step at a playback timeestablished by the user or at the default playback time. The followingis an example of playback times.

0: Not played back at all (skipped);

X: For example, 15 seconds or 30 seconds (0<X<FULL);

FULL: Played back at normal speed

In a supplemental step, the video clip categorized as a supplementalvideo clip may be played back in digest form. The following are examplesof playback in digest form.

(1) Screenshots are taken of the video clip associated with thesupplemental step at a predetermined interval (such as 10 seconds), andthe screenshots are played back;

(2) Screenshots are taken of the video clip associated with thesupplemental step at a predetermined interval (such as 10 seconds), thescreenshots are summarized in one or more images, and the summarizedimages are displayed or played back for a predetermined period of time(such as 60 seconds);(3) When chapters are to be skipped through in a video clip associatedwith a supplemental step, several seconds or several dozen seconds (forexample, 5 seconds) at the start of each chapter are linked together andplayed back; or(4) The video clip is played back at a speed faster than 1× (such as 2×,4× or 6×).

FIG. 4A and FIG. 4B show an example of a user interface of video clipscategorized as transitional steps in an embodiment of the presentinvention.

The screen (401) shown in FIG. 4A shows an example of the video clipscategorized as transitional step 1 (385) in the track (371) categorizedas shown in FIG. 3C. The video clips categorized as transitional stepsare played back during a transition from a supplemental step to a mainstep, from a main step to a supplemental step, or from a supplementalstep to another supplemental step.

Window (402) is displayed inside screen (401) to allow the user toconfirm whether or not the transitional step is to be displayed. Window(402) includes a “continue” button for playing back Step 2-1 throughStep 2-10, and a “skip” button for skipping playback of Step 2-1 throughStep 2-10.

Screen (401) also includes a slider (403) for controlling the playbackspot in the video clips associated with Steps 2-1 through Step 2-10(351), a start button (404) for starting the playback of the videoclips, a rewind button (405) for playing back a video clip again, and afast forward button (406) for fast forwarding through the playback ofthe video clips.

Screen (411) shown in FIG. 4B includes a slider bar (417) that can beused to adjust the playback speed of the video clips to, for example,0.5×, 1×, 2×, 4× or 8×. The user may adjust the slider bar (417) tochange the playback speed of the video clips to the desired speed.

Screen (421) shown in FIG. 4B has a pull-down menu (427) allowing theuser to set the playback of the video clips to “slow pace mode” (forexample, playback at 0.5×), “normal mode” (for example, playback at 1×),and “speed mode” (for example, playback at 2×). The user may select anymode from the pull-down menu (427) to play back the video clips usingthe desired mode.

Screen (431) shown in FIG. 4B has a scroll menu (437) allowing the userto select a specific step among Step 2-1 through Step 2-10 (351) onwhich Step 3-3 through Step 3-10 are premised. The user may select oneor more items from the scroll menu (437) in order to play back videoclips associated with the desired item.

FIG. 5A shows an example of video clips associated with user-selecteditems and dependent items and a video composed of the video clips in anembodiment of the present invention.

The categorized track (371) is the same categorized track (371) shown inFIG. 3C. In the categorized track (371), video clips are associated witheach of the categorized steps: transitional step 1 (385), supplementalstep 1 (383), transitional step 2 (386), main step 1 (381), transitionalstep 3 (387), supplemental step 2 (384), transitional step 4 (388), andmain step 2 (382).

In FIG. 5A, the respective video clips associated with transitional step1 (385), supplemental step 1 (383), transitional step 1 (386), main step1 (381), transitional step 3 (387), supplemental step 2 (384),transitional step 4 (388) and main step 2 (382) are transitional step 1video clip (511), supplemental step 1 video clip (512), transitionalstep 2 video clip (513), main step 1 video clip (514), transitional step3 video clip (515), supplemental step 2 video clip (516), transitionalstep 4 video clip (517) and main step 2 video clip (518). The electronicdevice may dynamically create a video by linking together transitionalstep 1 video clip (511), supplemental step 1 video clip (512),transitional step 2 video clip (513), main step 1 video clip (514),transitional step 3 video clip (515), supplemental step 2 video clip(516), transitional step 4 video clip (517) and main step 2 video clip(518). When the electronic device is a user terminal, the electronicdevice may play back the created video.

FIG. 5B shows an example in which a portion of the video clipsconstituting the video and categorized as supplemental steps is deletedin an embodiment of the present invention.

In response to an instruction to increase the playback speed of videoclips categorized as supplemental steps or in response to a settingincreasing the playback speed of video clips categorized as supplementalsteps, the electronic device may delete a certain number of sections(scenes) from video clips associated the supplemental steps (that is,supplemental step 1 video clip (512) and supplemental step 2 video clip(516)). In supplemental step 1 video clip (512) and supplemental step 2video clip (516), the deleted portions are the filler portions.

Supplemental step 1 video clip (522) and supplemental step 2 video clip(526) denote the same video clips after the filler portions have beendeleted from supplemental step 1 video clip (512) and supplemental step2 video clip (516).

The electronic device may dynamically create a video by linking togethertransitional step 1 video clip (511), supplemental step 1 video clip(522), transitional step 2 video clip (513), main step 1 video clip(514), transitional step 3 video clip (515), supplemental step 2 videoclip (526), transitional step 4 video clip (517) and main step 2 videoclip (518). When the electronic device is a user terminal, theelectronic device may play back the created video. When the electronicdevice is a user terminal, the electronic device may play back thecreated video.

FIG. 5C shows an example in which video clips constituting the video andcategorized as supplemental steps are played back at a speed faster thanvideo clips categorized as main steps or transitional steps in anembodiment of the present invention.

In response to an instruction to increase the playback speed of videoclips categorized as supplemental steps or in response to a settingincreasing the playback speed of video clips categorized as supplementalsteps, the electronic device sets the playback of supplemental step 1video clip (522) and supplemental step 2 video clip (526) to a speed(532, 536) faster than the playback speeds (531, 533, 534, 535, 537,respectively) of the other video clips (transitional step 1 video clip(511), transitional step 2 video clip (513), main step 1 video clip(514), transitional step 3 video clip (515), transitional step 4 videoclip (517) and main step 2 video clip (518)).

FIG. 6A through FIG. 6C and FIG. 7A through FIG. 7C show embodimentsother than the embodiment shown in FIG. 3A through FIG. 3C.

FIG. 6A shows an example in which dependent items having a dependentrelationship with user-selected items are identified in an embodiment ofthe present invention. This embodiment is different from the one shownin FIG. 3A.

In the step ID list (601) shown in FIG. 6A, the items selected by theuser in FIG. 2C through FIG. 2D, that is, Step 3-3 through Step 6-6(611) and Step 8-2 through Step 9-3 (612), are linked.

The electronic device may identify which dependent items have adependent relationship with the items in the step ID list (601) usingthe dependent item identifying table (621). In the dependent itemidentifying table (621), the steps with step IDs 3-3 through 3-10 have adependent relationship with the steps having step IDs 2-x (that is, Step2-1 through Step 2-10). Also, in the dependent item identifying table(621), the steps with step IDs 9-1 through 9-10 have a dependentrelationship with the steps having step IDs 10-x and 7-x (that is, Step10-1 through Step 10-10 and Step 7-1 through Step 7-10). The dependentitem identifying table (621) may assume the format of a table or anyother data format as long as step IDs are associated with dependencyIDs.

The electronic device uses the dependency item identifying table (621)to determine that Step 3-3 through Step 3-10 among Step 3-3 through Step6-6 (611) have a dependency relationship with Steps 2-x (that is, Step2-1 through Step 2-10) (631).

Similarly, the electronic device uses the dependency item identifyingtable (621) to determine that Step 9-1 through Step 9-3 among Step 8-2through Step 9-3 (612) have a dependency relationship with Steps 10-xand Steps 7-x (that is, Step 10-1 through Step 10-10 and Step 7-1through Step 7-10) (632). Because the dependency IDs in the dependentitem identifying table (621) are listed in the order 10-x then 7-x, theelectronic device determines that Step 10-1 through Step 10-10 are itemspreceding Step 7-1 through Step 7-10 (632). These are preceding itemsbecause the video clips associated with Step 10-1 through Step 10-10 areplayed back prior to the video clips associated with Step 7-1 throughStep 7-10.

Each of Step 2-1 through Step 2-10 (651), and Step 10-1 through Step10-10 and Step 7-1 through Step 7-10 (652) identified as having adependent relationship are inserted by the electronic device into thestep ID list (601) according to, for example, the flowchart for creatinga step ID list shown in FIG. 9B through FIG. 9D in order to create stepID list (641). Step ID list (641) has Step 2-1 through Step 2-10 (651),Step 3-3 through Step 6-6 (611), Step 8-2 through Step 8-10 (612 a),Step 10-1 through Step 10-10 and Step 7-1 through Step 7-10 (652), andStep 9-1 through Step 9-3 (612 b). Step 10-1 through Step 10-10 and Step7-1 through Step 7-10 (652) are listed in the order Step 10-1 throughStep 10-10 and Step 7-1 through Step 7-10 (652) in the step ID list(641) and the dependency IDs are listed in the order 10-x and 7-x in thedependent item identifying table (621) because Step 10-1 through Step10-10 are items that precede Step 7-1 through Step 7-10. In thisexample, Step 2-1 through Step 2-10 (651) are inserted before Step 3-3through Step 6-6 (611), and Step 10-1 through Step 10-10 and Step 7-1through Step 7-10 (652) are inserted between Step 8-2 through Step 8-10(612 a) and Step 9-1 through Step 9-3 (612 b) among Step 8-2 throughStep 9-3 (612).

Alternatively, the electronic device may create a step ID list (notshown) by inserting items specified as having a dependent relationshipwith Step 3-3 through Step 3-10 and Step 9-1 through Step 9-3, that is,Step 2-1 through Step 2-10 as well as Step 10-1 through Step 10-10 andStep 7-1 through Step 7-10, into step ID list (601) and then sorting theitems by step number or in reference to dependent relationship flagsindicating the step with which there is a dependent relationship.

FIG. 6B shows an example in which the video clips associated withuser-selected items and dependent items in step ID list (641) areidentified in an embodiment of the present invention. This embodiment isdifferent from the one shown in FIG. 3B.

The electronic device may identify the video clips associated with eachitem in the step ID list (641) using video clip identifying table (661)or video clip identifying table (662).

The electronic device uses the video clip identifying table (661) toacquire the video clip ID or script ID associated with the step IDcorresponding to a step number in the step ID list (641), and associatesthe acquired video clip ID or script ID with the step in the step IDlist (641). The electronic device can dynamically create a provisionalvideo by linking the associated video clips or scripts.

Similarly, the electronic device uses the video clip identifying table(662) to acquire the video clip ID or script ID associated with the stepID corresponding to a step number in the step ID list (641), andassociates the acquired video clip ID or script ID with the step in thestep ID list (641). The electronic device can dynamically create aprovisional video by linking the associated video clips or scripts.

FIG. 6C shows an example in which the video clips in a video areclassified by category (“main step”, “transitional step”, supplementalstep”). This embodiment is different from the one shown in FIG. 3C.

The step ID list (641) shown in FIG. 6C is the same step ID list (641)shown in FIG. 6B, and has Step 2-1 through Step 2-10 (651), Step 3-3through Step 6-6 (611), Step 8-2 through Step 8-10 (612 a), Step 7-1through Step 7-10 (652) and Step 9-1 through Step 9-3 (612 b). Videoclips are associated with each step using the process shown in FIG. 6B.

The electronic device sets the category of the video clips associatedwith items in the step ID list (641) identified by the user (Step 3-3through Step 8-10 (612 a) and Step 9-1 through Step 9-3 (612 b)) to“main step”.

The electronic device sets the category of the video clips associated inthe step ID list (641) with dependent items identified as having adependent relationship with user-specified items (Step 2-1 through Step2-10 (651), and Step 10-1 through Step 10-10 and Step 7-1 through Step7-10 (652)) to “supplemental step”.

The electronic device sets the category of the video clips associatedwith items directly adjacent to items and dependent items in the step IDlist (641) identified by the user (the set Step 2-10 through Step 3-3(654), the set Step 8-10 through Step 10-1 (655), and the Step 7-10through Step 9-1 (656)) to “transitional step”. When the beginning stepin the step ID list (641) is a supplemental step, the category of thevideo clip associated with Step 2-1 (657) is also set to “transitionalstep” because a transitional step has to be played at the video startpoint. A transitional step is a portion corresponding to a scene change.When a video clip is set as a transitional step, the transitional stepmay be added to a supplemental step/main step/main step/supplementalstep combination or supplemental step/supplemental step combination.When the beginning of the step ID list (641) is a supplemental step, itmay be a transitional step (the start step played at the video startpoint)/supplemental step combination.

In the categorized track (671), the two categories of supplemental stepand transitional step have been applied, respectively, to the videoclips associated with the beginning of a supplemental step (Step 2-1,Step 10-1), and to the video clips associated with the end of asupplemental step (Step 2-10, Step 7-10). Similarly, when a main stepand a supplemental step are adjacent to each other, the two categoriesof main step and transitional step have been applied, respectively, tothe video clips associated with the beginning of a main step (Step 3-3,Step 9-1), and to the video clip associated with the end of a main step(Step 8-10).

In the categorized track (671), the video clips associated with eachitem in the categorized track (671) were categorized as follows:transitional step 1 (685), supplemental step 1 (683), transitional step2 (686), main step 1 (681), transitional step 3 (687), supplemental step2 (684), transitional step 4 (688), and main step 2 (682).

As in the case of FIG. 5A, a video clip is associated with transitionalstep 1 (685), supplemental step 1 (683), transitional step 2 (686), mainstep 1 (681), transitional step 3 (687), supplemental step 2 (684),transitional step 4 (688), and main step 2 (682) in the categorizedtrack (671). Therefore, the electronic device can dynamically create avideo by linking together transitional step 1 (685), supplemental step 1(683), transitional step 2 (686), main step 1 (681), transitional step 3(687), supplemental step 2 (684), transitional step 4 (688), and mainstep 2 (682). When the electronic device is a user terminal, theelectronic device may play back the created video.

By categorizing each video clip (using “main step”, “transitional step”and “supplemental step”), the video clips can be played back by categoryin different ways.

FIG. 7A shows an example in which dependent items having a dependentrelationship with user-selected items are identified in an embodiment ofthe present invention. This embodiment is different from the ones shownin FIG. 3A and FIG. 6A.

In the step ID list (701) shown in FIG. 7A, the items selected by theuser in FIG. 2C through FIG. 2D, that is, Step 3-3 through Step 6-6(711) and Step 8-2 through Step 9-3 (712), are linked.

The electronic device may identify which dependent items have adependent relationship with the items in the step ID list (701) usingthe dependent item identifying table (721). The dependent itemidentifying table (721) may assume the format of a table or any otherdata format as long as step IDs are associated with dependency IDs. Thedependent item identifying table (721) may define dependent itemsrecursively, for example, user-selected items, items dependent onuser-selected items, items dependent on dependent items, etc.

In the dependent item identifying table (721), the steps with step IDs3-3 through 3-10 have a dependent relationship with the steps havingstep IDs 2-x (that is, Step 2-1 through Step 2-10). Also, in thedependent item identifying table (621), the steps with step IDs 9-1through 9-10 have a dependent relationship with the steps having stepIDs 7-x (that is, Step 7-1 through Step 7-10). Similarly, the steps withstep IDs 2-x (that is, Step 2-1 through Step 2-10), which are dependenton the steps with step IDs 3-3 through 3-10, in turn have a dependentrelationship with the steps having step IDs 10-x (that is, Step 10-1through Step 10-10).

The electronic device uses the dependency item identifying table (721)to determine that Step 3-3 through Step 3-10 among Step 3-3 through Step6-6 (711) have a dependency relationship with Steps 2-x (that is, Step2-1 through Step 2-10) (731).

Similarly, the electronic device uses the dependency item identifyingtable (721) to determine that Step 9-1 through Step 9-3 among Step 8-2through Step 9-3 (712) have a dependency relationship with Steps 7-x(that is, Step 7-1 through Step 7-10) (732).

Similarly, the electronic device uses the dependency item identifyingtable (721) to determine that Steps 2-1 through Step 2-10 have adependency relationship with Steps 10-x (that is, Step 10-1 through Step10-10) (733).

The electronic device may insert “Step 2-1 through Step 2-10” and “Step10-1 through Step 10-10 and Step 7-1 through Step 7-10” identified ashaving a dependency relationship into step ID list (701), and then sorteach step by a predetermined reference such as dependency relationshipflags (722) showing the dependency relationships of the steps to createa step ID list (741). The dependency relationship flags (722) are usedto sort steps. For example, they are used to sort the steps in orderfrom the leaves of the graph shown in FIG. 7A (using a so-calleddepth-first search).

In the step ID list (741), Step 10-1 through Step 10-10 (751) are sortedprior to Step 2-1 through Step 2-10 (752) because Steps 10-x precedeSteps 2-x according to their relationship in the dependency relationshipflags (722). Similarly, Step 2-1 through Step 2-10 (752) are sortedprior to Step 3-3 through Step 3-10 (711) because Steps 2-x precedeSteps 3-x according to their relationship in the dependency relationshipflags (722). Similarly, Step 7-1 through Step 7-10 (753) are sortedprior to Step 9-1 through Step 9-3 (712) because Steps 7-x precede Steps9-x according to their relationship in the dependency relationship flags(722).

In the step ID list (741), data (such as a flag) is appended to eachstep ID indicating whether the step was selected by the user or insertedas a result of resolving a dependency relationship with a user-selectedstep. In the step ID list (741), the data may be used to differentiatenon-underlined steps (711 and 712) and underlined steps (751, 752, 753).

As shown in the step ID list (741), the electronic device may insert atransitional step (for example, “T: task ID”) before and after a stepinserted as a result of resolving a dependency relationship(corresponding to a supplemental step). For example, the transitionalsteps in the step ID list (741) are [T:10-x, 2-x] and [T:3-x], and[T:7-x] and [T:9-x]. Here, [T:10-x, 2-x] is a transitional step insertedat the beginning of the “supplemental step” category appended to videoclips associated with Step 10-1 through Step 10-10 and Step 2-1 throughStep 2-10. Similarly, [T:3-x] is a transitional step inserted at the endof the “supplemental step” category appended to video clips associatedwith Step 10-1 through Step 10-10 and Step 2-1 through Step 2-10.Similarly, [T:7-x] is a transitional step inserted at the beginning ofthe “supplemental step” category appended to video clips associated withStep 7-1 through Step 7-10. Similarly, [T:9-x] is a transitional stepinserted at the end of the “supplemental step” category appended tovideo clips associated with Step 7-1 through Step 7-10. The transitionalsteps inserted at the end of the “supplemental step” category (forexample, [T:3-x] and [T:9-x]) are used as steps indicating a transitionto a main step (that is, a clip notifying the user of a transition to amain step).

Alternatively, the electronic device may insert a place holder (forexample, the set [5: task ID] and [E: task ID]) before and after a stepinserted as a result of resolving a dependency relationship in order toindicate the location at which a transitional step video clip is to beinserted. In [5: task ID], “S” means the start of the inserted step. In[E: task ID], “E” means the end of the inserted step. The electronicdevice may insert [S:10-X] and then [E:10-X] before Step 10-1 throughStep 10-10 inserted as a result of resolving a dependency relationship,insert [S:2-X] and then [E:2-X] before Step 2-1 through Step 2-10inserted as a result of resolving a dependency relationship, and insert[S:7-X] and then [E:7-X] before Step 7-1 through Step 7-10 inserted as aresult of resolving a dependency relationship (not shown).

FIG. 7B shows an example in which the video clips associated withuser-selected items and dependent items in step ID list (741) areidentified in an embodiment of the present invention. This embodiment isdifferent from the one shown in FIG. 3B and FIG. 6B.

The electronic device may identify the video clips associated with eachitem in the step ID list (741) using video clip identifying table (761)or video clip identifying table (762).

The electronic device uses the video clip identifying table (761) toacquire the video clip ID or script ID associated with the step IDcorresponding to a step number in the step ID list (741), and associatesthe acquired video clip ID or script ID with the step in the step IDlist (741). The electronic device can dynamically create a provisionalvideo by linking the associated video clips or scripts.

Similarly, the electronic device uses the video clip identifying table(762) to acquire the video clip ID or script ID associated with the stepID corresponding to a step number in the step ID list (741), andassociates the acquired video clip ID or script ID with the step in thestep ID list (741). The electronic device can dynamically create aprovisional video by linking the associated video clips or scripts.

FIG. 7C shows an example in which the video clips in a video areclassified by category (“main step”, “transitional step”, supplementalstep”). This embodiment is different from the one shown in FIG. 3C.

The step ID list (741) shown in FIG. 7C is the same step ID list (741)shown in FIG. 7B, and has Step 10-1 through Step 10-10 and Step 2-1through Step 2-10 (751), Step 7-1 through Step 7-10 (752), Step 3-3through Step 6-6 (711), (753), and Step 8-2 through Step 9-3 (712).Video clips are associated with each step using the process shown inFIG. 7B.

The electronic device sets the category of the video clips associatedwith items in the step ID list (741) identified by the user (Step 3-3through Step 6-6 (711), Step 8-2 through Step 8-10 (712 a), and Step 9-1through Step 9-3 (712 b)) to “main step”.

The electronic device sets the category of the video clips associated inthe step ID list (741) with dependent items identified as having adependent relationship with user-specified items (Step 10-1 through Step10-10 and Step 2-1 through Step 2-10 (751), and Step 7-1 through Step7-10 (752)) to “supplemental step”.

The electronic device sets the category of the video clips associatedwith the transitional step inserted before and after a dependent item inthe step ID list (741) (that is, transitional step (754) and thentransitional step (755) inserted before the steps composed of Step 10-1through Step 10-10 and Step 2-1 through Step 2-10, and transitional step(756) and then transitional step (757) inserted before Step 7-1 throughStep 7-10) to “transitional step”. A transitional step is a portioncorresponding to a scene change.

In the categorized track (771), the video clips associated with eachitem in the categorized track (771) were categorized as follows:transitional step 1 (785), supplemental step 1 (783), transitional step2 (786), main step 1 (781), transitional step 3 (787), supplemental step3 (784), transitional step 4 (788), and main step 2 (782).

As in the case of FIG. 5A, a video clip is associated with transitionalstep 1 (785), supplemental step 1 (783), transitional step 2 (786), mainstep 1 (781), transitional step 3 (787), supplemental step 3 (784),transitional step 4 (788), and main step 2 (782) in the categorizedtrack (771). Therefore, the electronic device can dynamically create avideo by linking together transitional step 1 (785), supplemental step 1(783), transitional step 2 (786), main step 1 (781), transitional step 3(787), supplemental step 3 (784), transitional step 4 (788), and mainstep 2 (782). When the electronic device is a user terminal, theelectronic device may play back the created video.

By setting each video clip according to category (“main step”,“transitional step”, “supplemental step”), a video can be played back indifferent ways by category. The following are examples of playback bycategory.

FIG. 8 shows the timing for the process of creating a video from videoclips (the video clip consolidation process) and the process of playingback the video clips in the video.

When the user has finished selecting items (811), the electronic deviceresponds by starting the creation of a video (812). In response tohaving started to create a video (812), the electronic device beginsplaying back the created video (821) before ending the creation of thevideo (813). In other words, the electronic device creates a video andplays back the video in parallel.

FIG. 9A through FIG. 9F shows an example of a flowchart of the processfor dynamically creating a video on the basis of a structured document.In the following explanation, the main unit (electronic device)performing each step shown in FIG. 9A through FIG. 9F depends on whetherthe electronic device is a user terminal or the electronic device is aserver.

FIG. 9A shows an example of a flowchart of the entire process performedto dynamically create a video on the basis of a structured document inan embodiment of the present invention.

When the electronic device is a user terminal

In Step 901, the user terminal starts the process of dynamicallycreating a video on the basis of a structured document.

In Step 902, the user terminal retrieves an electronic manual frommemory (103, 123) in response to a user request, and displays theretrieved electronic manual on the display device (106, 126) of the userterminal. The user terminal may display the electronic document on theleft side of the application window or in a different applicationwindow.

In Step 903, the user terminal switches to the selection screen forselecting one or more items from the electronic manual in response tothe user having clicked on the “switch to selection screen” button. Theuser then selects one or more items in the selection screen. The userterminal identifies the items selected by the user as input, and uses adependent item identifying table (321, 621 or 721) to identify at leastone dependent item having a dependency relationship with an identifieditem. The user terminal then outputs the user-selected items and thedependent items as a step ID list. Step 903 is described in greaterdetail in FIG. 9B through FIG. 9D.

In Step 904, the user terminal uses a video clip identifying table (361,362, 661, 662, 761 or 762) to select at least one video clip associatedwith each item in the step ID list created in Step 903, and outputs thevideo clips as a track. Step 904 is described in greater detail in FIG.9E.

In Step 905, the user terminal creates a video from the track created inStep 904. Step 905 is described in greater detail in FIG. 9F.

In Step 906, the user terminal may provide a video or information forplaying back a video to the user.

In Step 907, the user terminal may optionally output the video clips inthe video from a storage device (108, 109, 128) in response to a videocreation instruction from the user or may download the video clips inthe video via a network (115, 135). In other words, the video clips inthe video may be retrieved by the user terminal from memory (103, 123).The user terminal then successively plays back the video retrieved frommemory (103, 123) on the display device (106, 126) of the user terminal.

In Step 908, the user terminal ends the process of dynamically creatinga video on the basis of a structured document.

When the electronic device is a server.

In Step 901, the user terminal starts the process of dynamicallycreating a video on the basis of a structured document.

In Step 902, the user terminal retrieves an electronic manual frommemory (103, 123) in response to a user request, and displays theretrieved electronic manual on the display device (106, 126) of the userterminal.

In Step 903, the user terminal switches to the selection screen forselecting one or more items from the electronic manual in response tothe user having clicked on the “switch to selection screen” button. Theuser then selects one or more items in the selection screen. The userterminal identifies the items selected by the user as input, and uses adependent item identifying table (321, 621 or 721) to identify at leastone dependent item having a dependency relationship with an identifieditem. The user terminal then sends the information on the user-selecteditems and identified dependent items to a server connected to the userterminal via a network. Alternatively, the user terminal may identifythe user-selected items and send information on the identified items toa server connected to the user terminal via a network. The serverreceives the user-selected items and the identified dependent items fromthe user terminal. Alternatively, the server receives information on theuser-selected items from the user terminal and uses a dependent itemidentifying table (321, 621 or 721) to identify at least one dependentitem having a dependency relationship with a user-selected item in thereceived information. The server then outputs the user-selected itemsand the dependent items as a step ID list. Step 903 is described ingreater detail in FIG. 9B through FIG. 9D.

In Step 904, the server uses a video clip identifying table (361, 362,661, 662, 761 or 762) to select at least one video clip associated witheach item in the step ID list created in Step 903, and outputs the videoclips as a track. Step 904 is described in greater detail in FIG. 9E.

In Step 905, the server creates a video from the track created in Step904. Step 905 is described in greater detail in FIG. 9F.

In Step 906, the server sends to the user terminal video data for thevideo created in Step 905 or information on playing back the video (forexample, information indicating the location where the video data isbeing stored such as a URL, URI or file name).

In Step 907, the user terminal may optionally receive video data orinformation on playing back the video data from the server in responseto a video creation instruction from the user. When the user terminalreceives the video data, the user terminal retrieves the received videofrom a storage device (108, 109, 128), and stores the video clips in thevideo in memory (103, 123). When the user terminal receives informationon playing back the video data, the user terminal downloads the videodata via the network (115, 135), retrieves the downloaded video from thestorage device (108, 109, 128), and stores the video clips in the videoin memory (103, 123). The user terminal then successively plays back theretrieved video in memory (103, 123) on the display device (106, 126) ofthe user terminal.

In Step 908, the user terminal ends the process of dynamically creatinga video on the basis of a structured document.

FIG. 9B shows an example of a flowchart for creating a step ID list inthe process of dynamically creating a video on the basis of a structureddocument according to an embodiment of the present invention.

When the electronic device is a user terminal

In Step 911, the user terminal begins the processing in Step 903 shownin FIG. 9A, that is, to acquire user-selected items and create a step IDlist.

In Step 912, the user terminal initializes a step ID list in order tocreate a step ID list. For example, the user terminal may initialize astorage device (such as a buffer) in order to store a step ID list. Thestep ID list in the flowchart is a global variable referenced in all ofthe recursive steps performed to identify step IDs (see FIG. 9C throughFIG. 9D).

In Step 913, the user terminal resolves the dependent items of eachuser-selected item, and creates a step ID list. In Step 913, the step IDlist is retrieved as an initialized global variable, and is outputted asa step ID list updated in a format including step IDs with dependencyflags affixed to indicate a dependent step. Step 913 is described ingreater detail in FIG. 9C.

In Step 914, the user terminal scans the step ID list with affixeddependency flags in order from the top, determines the locations atwhich transitional steps need to be inserted into the step ID, andinserts transitional steps at these locations. In Step 914, thetransitional step insertion process is treated as a sub-routine, and thestep ID list outputted in Step 913 is updated. Step 914 is described ingreater detail in FIG. 9D.

In Step 915, the user terminal outputs the step ID list.

In Step 916, the user terminal ends the process in Step 903 shown inFIG. 9A.

When the electronic device is a server

The processing performed in Step 911 through Step 916 when theelectronic device is a server is the same as the processing performed inStep 911 through Step 916 when the electronic device is a user terminal.Therefore, the main unit (user terminal) performing the processing inStep 911 through Step 916 is simply replaced by a server.

FIG. 9C shows an example of a flowchart for the process of resolvingdependent items and creating a step ID list in the flowchart forcreating a step ID list shown in FIG. 9B.

When the electronic device is a user terminal

In Step 921, the user terminal begins to perform the processing in Step913 shown in FIG. 9B, that is, to resolve the dependent items and createa step ID list.

In Step 922, the user terminal retrieves unprocessed user-selected itemsin order from the most recent number, and acquires the step ID of theretrieved items.

In Step 923, the user terminal determines whether or not certainconditions have been met, namely, whether the item retrieved in Step 922has a dependent item and whether the dependency relationship of the taskID belonging to the step ID of the retrieved item is unresolved. WhenStep 3-x (corresponding to the step ID) is dependent on Step 2-x asshown in the dependent item identifying table (721) in FIG. 7A, Step 3-3in the step ID list (701) is processed. After inserting Step 2-x in thestep ID list, Step 3-4 is evaluated. In order to insert Step 2-3 onceagain, the dependency relationship to a step belonging to the same taskID such as Step 3-x has to be resolved. Therefore, a step belonging tothe same task ID is not reevaluated. When the conditions have been met,the user terminal proceeds to Step 924. When the conditions have notbeen met, the user terminal proceeds to Step 926.

When the conditions have been met as determined by Step 923, the userterminal in Step 924 enters the dependent item of the retrieved item asinput, and recursively calls and executes the processing shown in FIG.9C to further update the step ID list. In other words, when it has beendetermined that there is an item dependent on a dependent item of theretrieved item and that the item is further dependent, the user terminalmay add the item that is even further dependent and update the step IDlist. Therefore, a recursive structure is employed in Step 924. In thisway, the user terminal is able to resolve items dependent even furtheron user-selected items.

In Step 925, the user terminal affixes a dependency flag to the step IDnewly added to the step ID list. In Step 924, when Step 924 isrecursively called and Step 924 is executed again, the step ID of thedependent item is added to the step ID list serving as a global variable(this is referred to in FIG. 10A as the “global list”). However, whenthe added step ID is for a dependent item, it has to be differentiated.In order to differentiate an added step ID for a dependent item, theuser terminal compares the lists before and after the processing in Step924 was performed, and affixes a flag indicating a dependent item to thenewly inserted item.

In Step 926, the user terminal inserts the step ID corresponding to theitem determined in Step 923 into the step ID list when the conditions inStep 923 are not met.

In Step 927, the user terminal determines whether there are anyunprocessed items. When an unprocessed item is present, the userterminal returns to Step 922. When an unprocessed item is not present,the user terminal proceeds to Step 928 in which the process is ended.

In Step 928, the user terminal ends the processing in Step 913 of FIG.9B. Next, the user terminal inserts the step ID to which a dependencyflag was affixed in Step 925, determines the insertion location in thestep ID list for the transitional step used to insert the step ID towhich a dependency flag was affixed in Step 926. The user terminal thenstarts processing performed in accordance with the flowchart in FIG. 9Din order to insert a transitional step into the determined insertionlocation.

When the electronic device is a server

The processing performed in Step 921 through Step 928 when theelectronic device is a server is the same as the processing performed inStep 921 through Step 928 when the electronic device is a user terminal.Therefore, the main unit (user terminal) performing the processing inStep 921 through Step 928 is simply replaced by a server.

FIG. 9D shows an example of a flowchart for the process of determiningthe insertion locations for transitional steps in the step ID listcreated in accordance with the processing shown in the flowchart of FIG.9C and inserting the transitional steps in the flowchart for creating astep ID list shown in FIG. 9B.

When the electronic device is a user terminal

In Step 931, the user terminal begins to perform the processing in Step914 shown in FIG. 9B, that is, to determine the insertion location oftransitional steps and insert the transitional steps.

In Step 932, the user terminal initializes a start insertion location,which is a variable for storing the location of a dependent step when astep ID without an affixed dependency flag is retrieved. For example,the user terminal may initialize a storage device (such as a buffer) forstoring a “start insertion location” variable. Because the “startinsertion location” is set to the initial state or another value, it canbe used to determine whether or not a string of dependent steps is to bestarted.

In Step 933, the user terminal retrieves an unprocessed step ID from thestep ID list outputted by the processing performed according to theflowchart in FIG. 9C, for example, in the order of arrangement.

In Step 934, the user terminal determines whether or not the step IDretrieved in Step 933 is for a dependent step. When the step ID is for adependent step, the user terminal proceeds to Step 935. When the step IDis not for a dependent step, the user terminal proceeds to Step 937.

When the step ID retrieved in Step 933 is for a dependent step, the userterminal determines in Step 935 whether or not the “start insertionlocation” variable is in the initial state. When the “start insertionlocation” variable is in the initial state, the user terminal proceedsto Step 936. When the “start insertion location” variable is not in theinitial state, the user terminal proceeds to Step 941.

Because the “start insertion location” variable is in the initial state,the user terminal in Step 936 determines that the position just beforethe step ID retrieved in Step 933 is the start insertion location.

When the step ID retrieved in Step 933 is not for a dependent step, theuser terminal in Step 937 determines that the “start insertion location”variable is not in the initial state. Because the “start insertionlocation” variable is not in the initial state, the user terminalproceeds to Step 938. When the “start insertion location” variable is inthe initial state, the user terminal proceeds to Step 941.

When the “start insertion location” variable is not in the initialstate, the user terminal in Step 938 creates a new transitional step,affixes information in the ID list from the “start insertion location”variable to just before the step ID, and inserts a new transition stepin the “start insertion location” variable. In other words, the userterminal adds information on the step ID which is passed on to Step 938when a new transitional step is inserted into the start insertionlocation.

In Step 939, the user terminal creates a new transitional step, affixesstep ID information, and inserts the new transitional step just beforethe step ID. In other words, the user terminal adds the step IDinformation retrieved in Step 933 when a new transitional step is to beinserted in an end position.

In Step 940, the user terminal initializes the “start insert location”variable.

In Step 941, the user terminal determines whether or not there is anunprocessed step ID in the step ID list outputted by the processingperformed in accordance with the flowchart in FIG. 9C. The user terminalreturns to Step 933 when an unprocessed step ID is found. The userterminal proceeds to Step 942, which ends the processing, when anunprocessed step ID is not found.

In Step 942, the user terminal ends the processing in Step 914 of FIG.9B.

When the electronic device is a server

The processing performed in Step 931 through Step 942 when theelectronic device is a server is the same as the processing performed inStep 931 through Step 942 when the electronic device is a user terminal.Therefore, the main unit (user terminal) performing the processing inStep 931 through Step 942 is simply replaced by a server.

FIG. 9E shows an example of a flowchart for selecting video clips andcreating a track in the process of dynamically creating a video on thebasis of a structured document according to an embodiment of the presentinvention.

When the electronic device is a user terminal

In Step 951, the user terminal begins to perform the processing in Step904 shown in FIG. 9A, that is, to select video clips and create a track.

In Step 952, the user terminal initializes a provisional track in orderto create a track. For example, the user terminal may initialize astorage device (such as a buffer) to store a provisional track.

In Step 953, the user terminal retrieves the first unprocessed step IDin the step ID list created in Step 903, and acquires video clipinformation associated with the retrieved step ID. When, for example,the step ID is for a main step or a supplemental step, the user terminalextracts information on the corresponding clip from the video clipidentifying table, and affixes the information on the extracted videoclip to the item (data linking the step ID list to attributes of a mainstep, supplemental step or transitional step). As a result of appendingvideo clip information, three data sets are affixed to the item: a stepID, attributes of the main step, supplemental step or transitional step,and video clip information.

In Step 954, the user terminal inserts in the provisional track the stepID retrieved in Step 953 combined with information acquired in Step 953(video clip information).

In Step 955, the user terminal determines whether or not an unprocessedstep ID is present in the step ID list created in Step 903. When anunprocessed step ID is present, the user terminal returns to Step 953.When an unprocessed step ID is not present, the user terminal proceedsto Step 956.

In Step 956, the user terminal outputs the provisional track as thetrack when no unprocessed step IDs are present.

In Step 957, the user terminal ends the processing performed in Step 904of FIG. 9A.

When the electronic device is a server

The processing performed in Step 951 through Step 957 when theelectronic device is a server is the same as the processing performed inStep 951 through Step 957 when the electronic device is a user terminal.Therefore, the main unit (user terminal) performing the processing inStep 951 through Step 957 is simply replaced by a server.

FIG. 9F shows an example of a flowchart for creating a video from atrack in the process of dynamically creating a video on the basis of astructured document according to an embodiment of the present invention.

When the electronic device is a user terminal

In Step 961, the user terminal begins to perform the processing in Step905 shown in FIG. 9A, that is, to create a video from the track.

In Step 962, the user terminal initializes a provisional video in orderto create a video. For example, the user terminal may initialize astorage device (such as a buffer) to store a provisional video.

In Step 963, the user terminal retrieves video clip informationassociated with the first unprocessed step ID in the track created inStep 904, and acquires a video clip corresponding to the video clipinformation.

In Step 964, the user terminal affixes attribute information for playingback each video clip. For example, data is affixed to each step ID inthe track indicating whether or not the item was selected by the user asdescribed in the explanation of Step 953. As a result, the user terminalmay affix this as attribute information on the video clip correspondingto the data (for example, this is a main step selected by the user).

In Step 965, the video terminal consolidates the video clips in theprovisional video.

In Step 966, the user terminal determines whether or not there is anunprocessed step ID in the track created in Step 904. When anunprocessed step ID is present, the user terminal returns to Step 963.When an unprocessed step ID is not present, the user terminal proceedsto Step 967.

In Step 967, when an unprocessed step ID is not present, the userterminal outputs the provisional video to the storage device (108, 128)as the video to be played. Alternatively, when an unprocessed step ID isnot present, the user terminal may output information for playing backvideo from the provisional video to the storage device (108, 128).

In Step 968, the user terminal ends the processing performed in Step 905of FIG. 9A.

When the electronic device is a server

The processing performed in Step 961 through Step 968 when theelectronic device is a server is the same as the processing performed inStep 961 through Step 968 when the electronic device is a user terminal.Therefore, the main unit (user terminal) performing the processing inStep 961 through Step 968 is simply replaced by a server.

FIG. 10A shows a step ID list that has been consolidated according tothe processing shown in the flowchart of FIG. 9C.

In the step ID list (1001) shown in FIG. 10A, each item selected by theuser in FIG. 2C through FIG. 2D, that is, Step 3-3 through Step 6-6(311) and Step 8-2 through Step 9-3 (312) are linked.

The electronic device performs processing in accordance with theflowchart in FIG. 9C on the step ID list (1001) in order to resolve thedependent items created in accordance with the flowchart in FIG. 9C, andcreate a step ID list. Step ID list (1011) through step ID list (1029)indicate what changes have occurred in the global variables in thecreation of the list. Step 923, Step 924, Step 925, Step 926 and Step927 correspond to the step numbers shown in FIG. 9C.

For example, the change from “10-1, . . . , 10-10, 2-1, . . . , 2-10” instep ID list (1019) to “10-1, . . . , 10-10, 2-1, . . . , 2-10” in stepID list (1020), when viewed from the top of the recursive process (thatis, the flow in which Step 3-3 is processed), means dependency flagshave been affixed to all items in “10-1, . . . , 10-10, 2-1, . . . ,2-10” of step ID (1019) to become newly added items (steps). The itemsalready including affixed dependency flags keep their affixed dependencyflags.

FIG. 10B shows the insertion of transitional steps into a step ID list(1029) with all dependencies resolved according to the processing shownin the flowchart of FIG. 9D.

The step ID list (1031) shown in FIG. 10B is the same as the step IDlist (1029) shown in FIG. 10A.

Step ID list (1031) through step ID list (1042) show how transitionalsteps are inserted into step ID list (1031) in accordance with theprocess shown in the flowchart of FIG. 9D. The down arrows “↓” in stepID list (1031) through step ID list (1042) are cursors showing the stepID retrieved in Step 933 of the flowchart shown in FIG. 9D. Theelectronic device includes two combinations of the position of thedownward arrow “↓” and a “start insertion location” variable todetermine the start position and end position of a string of dependentsteps, and inserts transitional step [T:10-x, 2-x] (1034), transitionalstep [T:3-x] (1035), transitional step [T:7-x] (1040) and transitionalstep [T:9-x] (1041) into the step ID list (1031) in sequential order.

FIG. 11A is a functional block diagram of an electronic devicepreferably including a hardware configuration according to FIG. 1 fordynamically creating video on the basis of a structured document inaccordance with an embodiment of the present invention in which theelectronic device is a user terminal.

The user terminal (1101) includes an item identifying means (1111), avideo creating means (1112) and a video providing means (1113).

The item identifying means (1111) identifies at least one item in thestructured document (1121) selected by the user, and at least onedependent item having a dependency relationship with the user-selecteditem. For example, the item identifying means (1111) may use thedependent item identifying table (1124) to identify dependent items.

Also, the item identifying means (1111) identifies at least onecombination of a user-selected item in a start position (a first item)and a user-selected item in an end position (a second item), andidentifies each item (third items) between the first item and the seconditem.

Also, the item identifying means (1111) determines whether or not thereis any dependent item which is dependent on the first item, the seconditem or a third item. When a dependent item is present, the dependentitem is identified.

The video creating means (1112) dynamically creates a video (1123) basedon at least one video clip (1122) associated with each of the itemsidentified by the item identifying means (1111) and at least one videoclip (1122) associated with each dependent item identified by the itemidentifying means (1111).

The video creating means (1112) sorts the first item, the second itemand the third items in the order of the items (for example, thenumerical order of the items, or the numerical order of items inaccordance with an order suitable for playing back video clipsassociated with the items), and dynamically creates a video (1123) inthe order of the items on the basis of the video clips (1122) associatedwith the first item, the second item and the third items.

Also, the video creating means (1112) sorts the first item, the seconditem and third items and dependent items in the order of the items (forexample, the numerical order of the items, or the numerical order ofitems in accordance with an order suitable for playing back video clipsassociated with the items), and dynamically creates a video (1123) inthe order of the items on the basis of the video clips (1122) associatedwith the first item, the second item, the third items and the dependentitems.

Also, the video creating means (1112) inserts supplemental video clips(1122) used to explain the next video clip (1122) to be presented to theuser between a video clip (1122) associated with a first item, a seconditem or a third item, and a video clip (1122) associated with thedependent item directly adjacent to the first item, second item or thirditem (that is, the dependent item just before or just after the item).

In addition, the video creating means (1112) affixes a first flag usedto play back video clips (1122) associated with the first item, seconditem and third items according to a first attribute to the playbackinformation for each video clip (1122) associated with the first item,second item and third items, and affixes a second flag used to play backvideo clips (1122) associated with dependent items according to a secondattribute other than the first attribute to the playback information foreach video clip (1122) associated with a dependent item. The firstattribute and the second attribute can be any attribute selected fromamong video clip (1122) playback time, video clip (1122) playback speed,an indicator denoting whether or not to play back the video clip (1122),and the size of the playback window for the video clip (1122).

The video creating means (1112) also determines whether or not there isan operation script associated with an identified item and/or dependentitem, and dynamically creates a video (1123), when an operation scriptis present, according to virtual execution screens executed by thescript using video clips (1122) associated with items by the operationscript.

Also, the video creating means (1112) uses the video clip identifyingtable (1175) to select video clips (1122) associated with items anddependent items identified by the item identifying means (1111).

The video providing means (1113) may play back the video (1123) createdby the video creating means (1112) or may download video clips (1122) inaccordance with information on playing back the video (for example,information indicating the location where the video data is being storedsuch as a URL, URI or file name) and playing back the video.

FIG. 11B is a functional block diagram of an electronic devicepreferably including a hardware configuration according to FIG. 1 fordynamically creating video on the basis of a structured document inaccordance with an embodiment of the present invention in which theelectronic device is a server connected to a user terminal.

The server (1131) is connected to a user terminal (1161) via a network(the internet or an intranet).

The server (1131) includes an item identifying means (1141), a videocreating means (1142) and a video providing means (1143).

The user terminal (1161) includes an item selecting means (1171), anditem sending means (1172), a video or video information receiving means(1173), and a video playing means (1174).

The item selecting means (1171) in the user terminal (1161) allows theuser to select at least one item from the structured document (1181).

The item sending means (1172) in the user terminal (1161) sendsuser-selected items to the server (1131).

The item identifying means (1141) in the server (1131) receives theuser-selected items sent by the item sending means (1172), andidentifies the items selected by the user in the structured document(1151, which is the same as structured document (1181) or a copy).

The item identifying means (1141) also identifies at least one dependentitem having a dependency relationship to a user-specified item. Forexample, the item identifying means (1141) may use the dependent itemidentifying table (1154) to identify dependent items.

Also, the item identifying means (1141) identifies at least onecombination of a user-selected item in a start position (a first item)and a user-selected item in an end position (a second item), andidentifies each item (third items) between the first item and the seconditem.

Also, the item identifying means (1141) determines whether or not thereis any dependent item which is dependent on the first item, the seconditem or a third item. When a dependent item is present, the dependentitem is identified.

The video creating means (1142) in the server sorts the first item, thesecond item and the third items in the order of the items (for example,the numerical order of the items, or the numerical order of items inaccordance with an order suitable for playing back video clipsassociated with the items), and dynamically creates a video (1153) inthe order of the items on the basis of the video clips (1152) associatedwith the first item, the second item and the third items.

Also, the video creating means (1142) sorts the first item, the seconditem and third items and dependent items in the order of the items (forexample, the numerical order of the items, or the numerical order ofitems in accordance with an order suitable for playing back video clipsassociated with the items), and dynamically creates a video (1153) inthe order of the items on the basis of the video clips (1152) associatedwith the first item, the second item, the third items and the dependentitems.

Also, the video creating means (1142) inserts supplemental video clips(1152) used to explain the next video clip (1152) to be presented to theuser between a video clip (1152) associated with a first item, a seconditem or a third item, and a video clip (1152) associated with thedependent item directly adjacent to the first item, second item or thirditem (that is, the dependent item just before or just after the item).

In addition, the video creating means (1142) affixes a first flag usedto play back video clips (1152) associated with the first item, seconditem and third items according to a first attribute to the playbackinformation for each video clip (1152) associated with the first item,second item and third items, and affixes a second flag used to play backvideo clips (1152) associated with dependent items according to a secondattribute other than the first attribute to the playback information foreach video clip (1152) associated with a dependent item. The firstattribute and the second attribute can be any attribute selected fromamong video clip (1152) playback time, video clip (1152) playback speed,an indicator denoting whether or not to play back the video clip (1152),and the size of the playback window for the video clip (1152).

The video creating means (1142) also determines whether or not there isan operation script associated with an identified item and/or dependentitem, and dynamically creates a video (1153), when an operation scriptis present, according to virtual execution screens executed by thescript using video clips (1152) associated with items by the operationscript.

Also, the video creating means (1142) uses the video clip identifyingtable (1155) to select video clips (1122) associated with items anddependent items identified by the item identifying means (1141).

The video providing means (1143) in the server (1131) sends the video(1153) created by the video creating means (1142) or information onplaying back the video (1153) (for example, information indicating thelocation where the video data is being stored such as a URL, URI or filename) to the user terminal (1161).

The video or video information receiving means (1173) in the userterminal (1161) receives the video (1153) or information on playing backthe video that was sent by the video providing means (1143) in theserver (1131).

The video playing means (1174) in the user terminal (1161) plays backthe video (1153) while receiving the video (1153) from the videoproviding means (1143) in the server (1131), or stores the video (1153)sent by the video providing means (1143) in the server (1131) in astorage device (108, 128), and then plays back the video (1183) storedin the storage device (108, 128).

The video playing means (1174) may also receive information on playingback a video from the video providing means (1143) in the server (1131),download (1182) the video clip (1152) using the information, and playthe video. Alternatively, it may receive information on playing back avideo from the video providing means (1143) in the server (1131),retrieve video clips (1182, which are the same as video clips (1153) orcopies) stored in the storage device (108, 128), and play the video.

Electronic manuals present operating instructions in text form similarto paper manuals, and as video (using, for example, animation ofoperating procedures). However, it takes time to play video, and it is apoor use of time to have to watch or skip through operations that theuser has already learned.

It is an object of the present invention to provide an electronic devicewhich allows a user to select one or more items in an electronic manual(such as the table of contents, a heading, a title or a step),dynamically selects the video clips associated with the items selectedby the user and with items dependent on these items, and provides thevideo to the user.

It is an object of the present invention to provide an electronic devicewhich allows a user to select one or more items in an electronic manual,dynamically selects the video clips associated with the items selectedby the user and with items dependent on these items, and provides thevideo to the user.

The present invention is a method for dynamically creating video basedon a structured document by associating video clips with items in astructured document. This method identifies at least one item in thestructured document selected by a user, identifies at least onedependent item having a dependent relationship with the identified item,dynamically creates a video on the basis of at least one video clipassociated with each identified item and at least one video clipassociated with each identified dependent item, and provides the createdvideo or information for playing back the created video to the user.

The present invention provides a technique for dynamically creatingvideo based on a structured document. This technique may be incorporatedinto a method, computer system, electronic device program, or electronicdevice program product for dynamically creating video based on astructured document.

A first aspect of the present invention is a method for dynamicallycreating video based on a structured document by associating video clipswith items in a structured document, the method executing in anelectronic device the steps of:

identifying at least one item in the structured document selected by auser;

identifying at least one dependent item having a dependent relationshipwith the identified item;

dynamically creating a video on the basis of at least one video clipassociated with each identified item, and at least one video clipassociated with each identified dependent item; and

providing the video or information for playing back the video to theuser.

In an embodiment of the present invention, the at least one item in thestructured document selected by the user may include the item at thestart point and the item at the end point in the structured documentindicated by the user.

In an embodiment of the present invention, the step of identifying theselected item may further comprise the steps of:

identifying at least one combination of an item at the selected startpoint (the “first item”) and an item at the selected end point (the“second item”); and

identifying each item (“third items”) between the first item and thesecond item.

In an embodiment of the present invention, the step of identifying theselected item may further comprise the steps of:

determining whether or not there is a dependent item dependent on thefirst item, the second item, or any third item; and

identifying the dependent item when the presence of a dependent item hasbeen determined.

In an embodiment of the present invention, the step of dynamicallycreating the video may comprise the steps of:

sorting the first item, the second item and any third items in order;and

dynamically creating a video in the order of the items on the basis ofthe video clips associated with each of the first item, second item andthird items.

In an embodiment of the present invention, the step of dynamicallycreating the video may comprise the steps of:

sorting the first item, the second item, the third items and thedependent items in order;

and;

dynamically creating a video in the order of the items on the basis ofthe video clips associated with each of the first item, second item,third items and dependent items.

In an embodiment of the present invention, the step of dynamicallycreating the video may comprise the step of inserting between a videoclip associated with a first item, a second item or a third item and avideo clip associated with a dependent item directly adjacent to thefirst item, second item or third item a supplemental video clipexplaining the video clip to be presented to the user next.

In an embodiment of the present invention, the step of dynamicallycreating the video may further comprise the steps of:

affixing a first flag used to play back according to a first attributeeach video clip associated with the first item, second item and thirditems to the playback information for each video clip associated withthe first item, second item and third items; and affixing a second flagused to play back according to a second attribute other than the firstattribute each video clip associated with a dependent item to theplayback information for each video clip associated with a dependentitem.

In an embodiment of the present invention, the first attribute and thesecond attribute may be any attribute selected from among the video clipplayback time, the video clip playback speed, an indicator denotingwhether or not to play back the video clip, and the size of the playbackwindow for the video clip.

In an embodiment of the present invention, the step of dynamicallycreating the video may further comprise the steps of:

determining whether or not there is an operation script associated withan identified item and/or dependent item; and

dynamically creating a video, when an operation script is present,according to the virtual execution screens executed by the script usingvideo clips associated with items by the operation script.

In an embodiment of the present invention, the structured document mayhave a tree structure, and each item may correspond to a node in thetree structure.

In an embodiment of the present invention, the electronic device may bea server able to receive from a user terminal an item selected by theuser.

In an embodiment of the present invention, the step of providing videoor video information to the user may further comprise the step ofsending video information or information for playing back the video fromthe server to the user terminal when the electronic device is a server.

In an embodiment of the present invention, the step of identifying anitem may further comprise the step of receiving the item selected by theuser when the electronic device is a server.

In an embodiment of the present invention, the electronic device may bea user terminal including a structured document.

In an embodiment of the present invention, the step of providing videoor video information to the user may further comprise the step ofplaying back the provided video or the step of playing back the videousing the provided information when the electronic device is a userterminal.

In an embodiment of the present invention, the step of identifying anitem may further comprise the step of identifying a selected item inresponse to the selection of at least one item in a structured documentby the user when the electronic device is a user terminal.

A second aspect of the present invention is an electronic device fordynamically creating video based on a structured document by associatingvideo clips with items in a structured document, the electronic devicecomprising:

an item identifying means for identifying at least one item in thestructured document selected by a user, and at least one dependent itemhaving a dependent relationship with the identified item;

a video creating means for dynamically creating a video on the basis ofat least one video clip associated with each identified item, and atleast one video clip associated with each identified dependent item; and

a video providing means for providing the video or information forplaying back the video to the user.

In an embodiment of the present invention, at least one item in thestructured document selected by the user may include the item at thestart point and the item at the end point in the structured documentindicated by the user.

In an embodiment of the present invention, the item identifying meansmay:

identify at least one combination of an item at the selected start point(the “first item”) and an item at the selected end point (the “seconditem”); and

identify each item (“third items”) between the first item and the seconditem.

In an embodiment of the present invention, the item identifying meansmay:

determine whether or not there is a dependent item dependent on thefirst item, the second item, or any third item; and

identify the dependent item when the presence of a dependent item hasbeen determined.

In an embodiment of the present invention, the video creating means may:

sort the first item, the second item and any third items in order; and

dynamically create a video in the order of the items on the basis of thevideo clips associated with each of the first item, second item andthird items.

In an embodiment of the present invention, the video creating means may:

sort the first item, the second item, the third items and the dependentitems in order; and

dynamically create a video in the order of the items on the basis of thevideo clips associated with each of the first item, second item, thirditems and dependent items.

In an embodiment of the present invention, the video creating means mayinsert between a video clip associated with a first item, a second itemor a third item and a video clip associated with a dependent itemdirectly adjacent to the first item, second item or third item asupplemental video clip explaining the video clip to be presented to theuser next.

In an embodiment of the present invention, the video creating means may:

affix a first flag used to play back according to a first attribute eachvideo clip associated with the first item, second item and third itemsto the playback information for each video clip associated with thefirst item, second item and third items; and affix a second flag used toplay back according to a second attribute other than the first attributeeach video clip associated with a dependent item to the playbackinformation for each video clip associated with a dependent item.

In an embodiment of the present invention, the first attribute and thesecond attribute may each be any attribute selected from among the videoclip playback time, the video clip playback speed, an indicator denotingwhether or not to play back the video clip, and the size of the playbackwindow for the video clip.

In an embodiment of the present invention, the video creating means may:

determine whether or not there is an operation script associated with anidentified item and/or dependent item; and

dynamically create a video, when an operation script is present,according to the virtual execution screens executed by the script usingvideo clips associated with items by the operation script.

In an embodiment of the present invention, the structured document mayhave a tree structure, and each item may correspond to a node in thetree structure.

In an embodiment of the present invention, the electronic device may bea server able to receive from a user terminal an item selected by theuser.

In an embodiment of the present invention, the video providing means maysend video information or information for playing back the video fromthe server to the user terminal when the electronic device is a server.

In an embodiment of the present invention, the item identifying meansmay receive the item selected by the user when the electronic device isa server.

In an embodiment of the present invention, the electronic device may bea user terminal including a structured document.

In an embodiment of the present invention, the video creating means mayplay back the provided video or play back the video using the providedinformation when the electronic device is a user terminal.

In an embodiment of the present invention, the item identifying meansmay identify a selected item in response to the selection of at leastone item in a structured document by the user when the electronic deviceis a user terminal.

A third aspect of the present invention is an electronic system fordynamically creating video based on a structured document by associatingvideo clips with items in a structured document, the electronic systemcomprising:

a user terminal, and a server connected to the user terminal;

the user terminal sending to the server as a selected item at least oneitem in a structured document selected by the user; and

the server

receiving the item sent by the user terminal,

identifying the item in the structured document selected by the user,

identifying at least one dependent item having a dependent relationshipwith the selected item,

dynamically creating a video on the basis of at least one video clipassociated with each identified item and at least one video clipassociated with each identified dependent item, and

sending the video or information for playing back the video to the userterminal.

In an embodiment of the present invention, at least one item in thestructured document selected by the user may include the item at thestart point and the item at the end point in the structured documentindicated by the user.

In an embodiment of the present invention, the server may:

identify at least one combination of an item at the selected start point(the “first item”) and an item at the selected end point (the “seconditem”); and

identify each item (“third items”) between the first item and the seconditem.

In an embodiment of the present invention, the server may:

determine whether or not there is a dependent item dependent on thefirst item, the second item, or any third item; and

identify the dependent item when the presence of a dependent item hasbeen determined.

In an embodiment of the present invention, the server may:

sort the first item, the second item and any third items in order; and

dynamically create a video in the order of the items on the basis of thevideo clips associated with each of the first item, second item andthird items.

In an embodiment of the present invention, the server may:

sort the first item, the second item, the third items and the dependentitems in order; and

dynamically create a video in the order of the items on the basis of thevideo clips associated with each of the first item, second item, thirditems and dependent items.

In an embodiment of the present invention, the server may insert betweena video clip associated with a first item, a second item or a third itemand a video clip associated with a dependent item directly adjacent tothe first item, second item or third item a supplemental video clipexplaining the video clip to be presented to the user next.

In an embodiment of the present invention, the server may:

affix a first flag used to play back according to a first attribute eachvideo clip associated with the first item, second item and third itemsto the playback information for each video clip associated with thefirst item, second item and third items; and affix a second flag used toplay back according to a second attribute other than the first attributeeach video clip associated with a dependent item to the playbackinformation for each video clip associated with a dependent item.

In an embodiment of the present invention, the first attribute and thesecond attribute may each be any attribute selected from among the videoclip playback time, the video clip playback speed, an indicator denotingwhether or not to play back the video clip, and the size of the playbackwindow for the video clip.

In an embodiment of the present invention, the server may:

determine whether or not there is an operation script associated with anidentified item and/or dependent item; and

dynamically create a video, when an operation script is present,according to the virtual execution screens executed by the script usingvideo clips associated with items by the operation script.

In an embodiment of the present invention, the structured document mayhave a tree structure, and each item may correspond to a node in thetree structure.

In an embodiment of the present invention, the server may send videoinformation or information for playing back the video to the userterminal.

In an embodiment of the present invention, the server may receive theitem selected by the user.

In an embodiment of the present invention, the user terminal mayidentify a selected item in response to the selection of at least oneitem in a structured document by the user.

In an embodiment of the present invention, the user terminal may playback the video provided by the server or play back the video using theinformation provided by the server.

In a fourth aspect of the present invention, an electronic deviceprogram or electronic device program product for dynamically creatingvideo on the basis of a structured document executes in an electronicdevice each step of the method. When the electronic device is a server,the electronic device program may be a program for a server. When theelectronic device is a user terminal, the electronic device program maybe a program for a user terminal.

The electronic device program according to the present embodiment can bestored on any computer-readable recording medium, including a flexibledisk, MO, CD-ROM, DVD, BD, hard disk device, USB memory, ROM, MRAM orRAM. The electronic device program can also be downloaded from anotherdata processing system connected to a communication network for storageon a recording medium, or can be copied from another recording medium.The program can also be compressed, divided into a plurality ofprograms, and recorded on one or more recording media. An electronicdevice program product embodying the present invention can take avariety of forms. The electronic device program product can include, forexample, the recording medium on which the electronic device program isstored or a transmission medium over which the electronic device programis transmitted.

The summary of the present invention provided above is not intended toenumerate all of the required features of the present invention, and itis to be understood that the present invention includes all combinationsand sub-combinations of these constituent elements.

It should also be clear that a person skilled in the art could conceiveof various changes such as combining a plurality of machines andallocating functions to these machines to embody the hardwareconfigurational elements of the information processing device used in anembodiment of the present invention. These changes are encompassedwithin the concept of the present invention. However, theseconfigurational elements are for illustrative purposes only, and all ofthe configurational elements are not essential configurational elementsof the present invention.

Also, the present invention can be realized by hardware, software, or acombination of hardware and software. When the present invention isexecuted by a combination of hardware and software, it is typicallyexecuted by a device in which the electronic device program has beeninstalled. Here, the electronic device program is downloaded to thememory of the device and executed. The electronic device program thencontrols the device and executes the processing of the presentinvention. This electronic device program can be configured from groupsof instructions expressed in any language, code or notation. The devicecan execute the specific functions in the group of instructionsimmediately, or after (1) converting the group of instructions intoanother language, code or notation, and/or (2) copying the group ofinstructions to another medium.

The technique according to an embodiment of the present inventionenables one or more specific items to be selected from an electronicmanual by the user. The user can then select a video clip associatedwith dependent items that have a dependent relationship with theuser-selected item in addition to a video clip associated with theuser-selected item. The selected video clips are then consolidated todynamically create a video, and the dynamically created video can beprovided to the user. As a result, the user can selectively view videoclips related to an item to be learned or viewed by the user or to anitem in which the user is interested along with video clips associatedwith dependent items having a dependent relationship with the item. Inother words, the technique according to an embodiment of the presentinvention dynamically creates video including not only video clipsrelated to an item to be learned or viewed by the user or to an item inwhich the user is interested but also video clips associated withdependent items having a dependent relationship with the item. Thetechnique according to an embodiment of the present invention enablesvideo to be viewed on demand that includes both video clips related toan item to be learned or viewed by the user or to an item in which theuser is interested along with video clips associated with dependentitems having a dependent relationship with the item to be learned orviewed by the user or with the item in which the user is interested.

Because the technique according to an embodiment of the presentinvention enables the user to indicate different ways that the videorelated to a dependent item can be played back in the dynamicallycreated video, the technique can reduce the impression on the user thatthe video related to the dependent item has caused the user to have toview extra video.

The technique according to an embodiment of the present invention candynamically execute an operation to be performed by the user (forexample, a mouse operation, keyboard operation or input operation) in asimulated manner to show the user how to perform the operation by usingvideo clips of video data obtained by executing an operation script onthe user terminal instead of a prerecorded video script.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A computer program product comprising a computerreadable storage medium having program code embodied therewith, whereinthe computer readable storage medium is not a transitory signal per se,and wherein the program code is executable by a processor to cause theprocessor to perform a method comprising: selecting, based on user inputto an interface displaying an online manual for a software application,a plurality of items comprising an initial item, an end item, and anintervening item in a table of contents pane; compiling the plurality ofitems in a step identifier (ID) list in an order of the initial item,the intervening item, and the end item; identifying a dependent itemassociated with the initial item by querying a dependent itemidentifying table; inserting the dependent item in the step ID listprior to the initial item; identifying a plurality of video clipsassociated with each of the initial item, the intervening item, the enditem, the dependent item, and items adjacent to the initial item and theend item using a video clip identifying table, wherein the video clipidentifying table includes, for respective items in the online manual, arespective Universal Resource Locator (URL) to a corresponding videoclip; ordering the plurality of video clips according to the step IDlist; categorizing the plurality of video clips, wherein video clipscorresponding to the initial item, the intervening item, and the enditem are categorized as main steps, wherein video clips associated withthe dependent item are categorized as supplemental steps, and whereinvideo clips associated with items adjacent to the initial item and theend item are categorized as transitional steps; and creating aninstructional video corresponding to the plurality of items selectedfrom the online manual for the software application, wherein theinstructional video displays: a first transitional video clip associatedwith items adjacent to the initial item; a first supplemental video clipassociated with the dependent item; a main video clip associated withthe initial item, the intervening item, and the end item; and a secondtransitional video clip associated with items adjacent to the end item.2. The computer program product of claim 1, wherein the dependent itemidentifying table comprises task identifiers, step identifiers, anddependency identifiers.
 3. The computer program product of claim 1,wherein the method further comprises: displaying the instructional videoon the interface, wherein the displaying includes: a window with acontinue button and a skip button; and a slider for controlling theplayback spot in the instructional video.
 4. A method comprising:selecting, based on user input to an interface displaying an onlinemanual for a software application, a plurality of items comprising aninitial item, an end item, and an intervening item in a table ofcontents pane; compiling the plurality of items in a step identifier(ID) list in an order of the initial item, the intervening item, and theend item; identifying a dependent item associated with the initial itemby querying a dependent item identifying table; inserting the dependentitem in the step ID list prior to the initial item; identifying aplurality of video clips associated with each of the initial item, theintervening item, the end item, the dependent item, and items adjacentto the initial item and the end item using a video clip identifyingtable, wherein the video clip identifying table includes, for respectiveitems in the online manual, a respective Universal Resource Locator(URL) to a corresponding video clip; ordering the plurality of videoclips according to the step ID list; categorizing the plurality of videoclips, wherein video clips corresponding to the initial item, theintervening item, and the end item are categorized as main steps,wherein video clips associated with the dependent item are categorizedas supplemental steps, and wherein video clips associated with itemsadjacent to the initial item and the end item are categorized astransitional steps; and creating an instructional video corresponding tothe plurality of items selected from the online manual for the softwareapplication, wherein the instructional video displays: a firsttransitional video clip associated with items adjacent to the initialitem; a first supplemental video clip associated with the dependentitem; a main video clip associated with the initial item, theintervening item, and the end item; and a second transitional video clipassociated with items adjacent to the end item.
 5. The method of claim4, wherein the dependent item identifying table comprises taskidentifiers, step identifiers, and dependency identifiers.
 6. The methodof claim 4, wherein the method further comprises: displaying theinstructional video on the interface, wherein the displaying includes: awindow with a continue button and a skip button; and a slider forcontrolling the playback spot in the instructional video.
 7. A systemcomprising: a processor; and a computer-readable storage medium storingprogram code, wherein the computer-readable storage medium is not atransitory signal per se, and wherein the program code, when executed bythe processor, is configured to cause the processor to perform a methodcomprising: selecting, based on user input to an interface displaying anonline manual for a software application, a plurality of itemscomprising an initial item, an end item, and an intervening item in atable of contents pane; compiling the plurality of items in a stepidentifier (ID) list in an order of the initial item, the interveningitem, and the end item; identifying a dependent item associated with theinitial item by querying a dependent item identifying table; insertingthe dependent item in the step ID list prior to the initial item;identifying a plurality of video clips associated with each of theinitial item, the intervening item, the end item, the dependent item,and items adjacent to the initial item and the end item using a videoclip identifying table, wherein the video clip identifying tableincludes, for respective items in the online manual, a respectiveUniversal Resource Locator (URL) to a corresponding video clip; orderingthe plurality of video clips according to the step ID list; categorizingthe plurality of video clips, wherein video clips corresponding to theinitial item, the intervening item, and the end item are categorized asmain steps, wherein video clips associated with the dependent item arecategorized as supplemental steps, and wherein video clips associatedwith items adjacent to the initial item and the end item are categorizedas transitional steps; and creating an instructional video correspondingto the plurality of items selected from the online manual for thesoftware application, wherein the instructional video displays: a firsttransitional video clip associated with items adjacent to the initialitem; a first supplemental video clip associated with the dependentitem; a main video clip associated with the initial item, theintervening item, and the end item; and a second transitional video clipassociated with items adjacent to the end item.
 8. The system of claim7, wherein the dependent item identifying table comprises taskidentifiers, step identifiers, and dependency identifiers.
 9. The systemof claim 7, wherein the method further comprises: displaying theinstructional video on the interface, wherein the displaying includes: awindow with a continue button and a skip button; and a slider forcontrolling the playback spot in the instructional video.